-
• #52
+1
-
• #53
So agree. Also everyone Iβve interacted with off LFGSs has been a proper stand up on the pedals guy. Donation in progress.
-
• #54
Beginning to suspect Velocio staged the whole thing for a couple of rounds of cocktails by the beach...
-
• #56
To echo everything everyone has said - really REALLY appreciate your knowledge and skill in dealing with the problem, and sorting it all out. Thank you!
-
• #57
Just caught up on all this, great work, thanks for your endless efforts.
-
• #58
Good job and sorry had to go through this while on holiday. Always the way, innit?
I'm due a donation anyway so will double-up.
-
• #59
I wondered why access was problematic.
Thank you for the explanation!
-
• #60
+
Bon Vacation!! -
• #62
Excellent work. Unfortunately I never really appreciated or understood the level of effort it can take to keep this platform alive. Thank you.
-
• #63
Thanks for all your work getting this ship back on course, much appreciated!
-
• #65
+1
-
• #66
Always appreciated. Itβs an extraordinary hive of advice, ramblings and useful classifieds. Donated.
-
• #67
A good reminder that this place doesn't run by itself
Agreed. Donated.
-
• #68
I noticed yesterday or the day before that some avatars were broken.
I agree with this btw, but I think what had happened is that the results of requests have been cached during the attack. So I'll need to purge all of the caches to make it work again.
I'm out and about in Heraklion today, but will purge caches when I'm back at the hotel later
-
• #69
Avatars should be fixed now, a CTRL+F5 on your side should resolve it if you are still seeing broken avatars.
-
• #70
I am going to try something... I want to make DDoS attacks even more pointless... and most of the last ones appeared to know which sites existed, but not which paths were valid... a lot of the requests triggered 404 not found for paths that would never be found... i.e.
/wp-content
, which is a Wordpress path.Given I do know which URLs are valid... I wonder whether I can just block all unknown paths.
In a regular expression this is a negative lookahead, so I'm thinking of writing something like this:
location ~* ^(\/(?!comments|microcosms)|\/)${ return 403; }
of course the only way to truly know is to try, and I'm relaxing on my vacation and this feels like a fun thing to try... it will probably take 30 minutes to ensure I have all possible valid URLs, and then 15 minutes to deploy into production and test it :) yes, I test in prod
-
• #71
There are 183 valid URLs on this platform:
/ /about/ /about/cookies/ /about/privacy/ /about/terms/ /api /api/v1 /api/v1/auth /api/v1/auth/[0-9A-Za-z]+ /api/v1/auth0 /api/v1/comments /api/v1/comments/[0-9]+ /api/v1/comments/[0-9]+/attachments /api/v1/comments/[0-9]+/attachments/[0-9A-Za-z]+ /api/v1/comments/[0-9]+/attachments/[0-9A-Za-z]+.[A-Za-z]+ /api/v1/comments/[0-9]+/attributes /api/v1/comments/[0-9]+/attributes/[0-9a-zA-Z_-]+ /api/v1/comments/[0-9]+/incontext /api/v1/conversations /api/v1/conversations/[0-9]+ /api/v1/conversations/[0-9]+/attributes /api/v1/conversations/[0-9]+/attributes/[0-9a-zA-Z_-]+ /api/v1/conversations/[0-9]+/lastcomment /api/v1/conversations/[0-9]+/newcomment /api/v1/events /api/v1/events/[0-9]+ /api/v1/events/[0-9]+/attendees /api/v1/events/[0-9]+/attendees/[0-9]+ /api/v1/events/[0-9]+/attendeescsv /api/v1/events/[0-9]+/attributes /api/v1/events/[0-9]+/attributes/[0-9a-zA-Z_-]+ /api/v1/events/[0-9]+/lastcomment /api/v1/events/[0-9]+/newcomment /api/v1/files /api/v1/files/[0-9A-Za-z]+ /api/v1/files/[0-9A-Za-z]+.[0-9A-Za-z]+ /api/v1/geocode /api/v1/hosts/[0-9a-zA-Z-.]+ /api/v1/huddles /api/v1/huddles/[0-9]+ /api/v1/huddles/[0-9]+/lastcomment /api/v1/huddles/[0-9]+/newcomment /api/v1/huddles/[0-9]+/participants /api/v1/huddles/[0-9]+/participants/[0-9]+ /api/v1/ignored /api/v1/legal /api/v1/legal/cookies /api/v1/legal/privacy /api/v1/legal/service /api/v1/legal/terms /api/v1/metrics /api/v1/microcosms /api/v1/microcosms/[0-9]+ /api/v1/microcosms/[0-9]+/attributes /api/v1/microcosms/[0-9]+/attributes/[0-9a-zA-Z_-]+ /api/v1/microcosms/[0-9]+/roles /api/v1/microcosms/[0-9]+/roles/[0-9]+/criteria /api/v1/microcosms/[0-9]+/roles/[0-9]+/criteria/[0-9]+ /api/v1/microcosms/[0-9]+/roles/[0-9]+/members /api/v1/microcosms/[0-9]+/roles/[0-9]+/profiles /api/v1/microcosms/[0-9]+/roles/[0-9]+/profiles/[0-9]+ /api/v1/microcosms/[0-9]+/roles/[0-9a-A-Z_-]+ /api/v1/microcosms/tree /api/v1/out/[2-9a-zA-Z]+ /api/v1/permission /api/v1/polls /api/v1/polls/[0-9]+ /api/v1/polls/[0-9]+/attributes /api/v1/polls/[0-9]+/attributes/[0-9a-zA-Z_-]+ /api/v1/polls/[0-9]+/lastcomment /api/v1/polls/[0-9]+/newcomment /api/v1/profiles /api/v1/profiles/[0-9]+ /api/v1/profiles/[0-9]+/attachments /api/v1/profiles/[0-9]+/attachments/[0-9A-Za-z]+ /api/v1/profiles/[0-9]+/attachments/[0-9A-Za-z]+.[A-Za-z]+ /api/v1/profiles/[0-9]+/attributes /api/v1/profiles/[0-9]+/attributes/[0-9a-zA-Z_-]+ /api/v1/profiles/options /api/v1/profiles/read /api/v1/reserved/[0-9a-zA-Z]+ /api/v1/resolve /api/v1/roles /api/v1/roles/[0-9]+ /api/v1/roles/[0-9]+/criteria /api/v1/roles/[0-9]+/criteria/[0-9]+ /api/v1/roles/[0-9]+/members /api/v1/roles/[0-9]+/profiles /api/v1/roles/[0-9]+/profiles/[0-9]+ /api/v1/search /api/v1/site /api/v1/site/[0-9]+/attributes /api/v1/site/[0-9]+/attributes/[0-9a-zA-Z_-]+ /api/v1/site/menu /api/v1/sites /api/v1/sites/[0-9]+ /api/v1/sites/[0-9]+/menu /api/v1/sites/[0-9]+/status /api/v1/trending /api/v1/updates /api/v1/updates/preferences /api/v1/updates/preferences/[0-9]+ /api/v1/users /api/v1/users/[0-9]+ /api/v1/users/batch /api/v1/watchers /api/v1/watchers/[0-9]+ /api/v1/watchers/delete /api/v1/watchers/patch /api/v1/whoami /auth0login/ /comments/[0-9]+/ /comments/[0-9]+/attachments/ /comments/[0-9]+/delete/ /comments/[0-9]+/edit/ /comments/[0-9]+/incontext/ /comments/[0-9]+/source/ /comments/create/ /compare/ /conversations/[0-9]+/ /conversations/[0-9]+/delete/ /conversations/[0-9]+/edit/ /conversations/[0-9]+/newest/ /dashboard/ /dashboard/sites/ /dashboard/sites/create/ /dashboard/sites/edit/[0-9]+ /developers/ /error/ /events/[0-9]+/ /events/[0-9]+/csv /events/[0-9]+/delete /events/[0-9]+/edit /events/[0-9]+/newest /events/[0-9]+/rsvp /faqs/ /favicon.ico /features/ /forbidden/ /geocode/ /headers/ /huddles/ /huddles/[0-9]+/ /huddles/[0-9]+/invite/ /huddles/[0-9]+/leave/ /huddles/[0-9]+/newest/ /huddles/create/ /ignore/ /ignored/ /login/ /logout/ /microcosms/ /microcosms/[0-9]+/ /microcosms/[0-9]+/create/conversation/ /microcosms/[0-9]+/create/event/ /microcosms/[0-9]+/create/microcosm/ /microcosms/[0-9]+/delete/ /microcosms/[0-9]+/edit/ /microcosms/[0-9]+/memberships/ /microcosms/[0-9]+/memberships/[0-9]+/api/ /microcosms/[0-9]+/memberships/[0-9]+/edit/ /microcosms/[0-9]+/memberships/create/ /microcosms/create/ /moderate/ /moderate/do/ /notfound/ /out/[2-9a-zA-Z]+ /profiles/ /profiles/[0-9]+/ /profiles/[0-9]+/edit/ /profiles/[0-9]+/patch/ /profiles/read/ /robots.txt /search/ /static/.* /terms/ /today/ /trending/ /unignore/ /updates/ /updates/december/ /updates/settings/ /watchers/
-
• #72
that didn't work.
but I have another idea... since I have to avoid the negative lookahead, and was having trouble allowing
/
to still work... I can instead try to use the nginxmap
functionality to create a map of known URLs https://nginx.org/en/docs/http/ngx_http_map_module.html#map -
• #73
Well that appears to have worked.
In the
http
part of the Nginx config:map $uri $known_uri { / 1; ~^/about/?$ 1; ~^/about/cookies/?$ 1; ~^/about/privacy/?$ 1; ~^/about/terms/?$ 1; /api 1; /api/v1 1; /api/v1/auth 1; ~^/api/v1/auth/[0-9A-Za-z]+$ 1; /api/v1/auth0 1; /api/v1/comments 1; ~^/api/v1/comments/[0-9]+$ 1; ~^/api/v1/comments/[0-9]+/attachments$ 1; ~^/api/v1/comments/[0-9]+/attachments/[0-9A-Za-z]+$ 1; ~^/api/v1/comments/[0-9]+/attachments/[0-9A-Za-z]+.[A-Za-z]+$ 1; ~^/api/v1/comments/[0-9]+/attributes$ 1; ~^/api/v1/comments/[0-9]+/attributes/[0-9a-zA-Z_-]+$ 1; ~^/api/v1/comments/[0-9]+/incontext$ 1; /api/v1/conversations 1; ~^/api/v1/conversations/[0-9]+$ 1; ~^/api/v1/conversations/[0-9]+/attributes$ 1; ~^/api/v1/conversations/[0-9]+/attributes/[0-9a-zA-Z_-]+$ 1; ~^/api/v1/conversations/[0-9]+/lastcomment$ 1; ~^/api/v1/conversations/[0-9]+/newcomment$ 1; /api/v1/events 1; ~^/api/v1/events/[0-9]+$ 1; ~^/api/v1/events/[0-9]+/attendees$ 1; ~^/api/v1/events/[0-9]+/attendees/[0-9]+$ 1; ~^/api/v1/events/[0-9]+/attendeescsv$ 1; ~^/api/v1/events/[0-9]+/attributes$ 1; ~^/api/v1/events/[0-9]+/attributes/[0-9a-zA-Z_-]+$ 1; ~^/api/v1/events/[0-9]+/lastcomment$ 1; ~^/api/v1/events/[0-9]+/newcomment$ 1; /api/v1/files 1; ~^/api/v1/files/[0-9A-Za-z]+$ 1; ~^/api/v1/files/[0-9A-Za-z]+.[0-9A-Za-z]+$ 1; ~^/api/v1/geocode$ 1; ~^/api/v1/hosts/[0-9a-zA-Z-.]+$ 1; ~^/api/v1/huddles$ 1; ~^/api/v1/huddles/[0-9]+$ 1; ~^/api/v1/huddles/[0-9]+/lastcomment$ 1; ~^/api/v1/huddles/[0-9]+/newcomment$ 1; ~^/api/v1/huddles/[0-9]+/participants$ 1; ~^/api/v1/huddles/[0-9]+/participants/[0-9]+$ 1; ~^/api/v1/ignored$ 1; ~^/api/v1/legal$ 1; /api/v1/legal/cookies 1; /api/v1/legal/privacy 1; /api/v1/legal/service 1; /api/v1/legal/terms 1; /api/v1/metrics 1; /api/v1/microcosms 1; ~^/api/v1/microcosms/[0-9]+$ 1; ~^/api/v1/microcosms/[0-9]+/attributes$ 1; ~^/api/v1/microcosms/[0-9]+/attributes/[0-9a-zA-Z_-]+$ 1; ~^/api/v1/microcosms/[0-9]+/roles$ 1; ~^/api/v1/microcosms/[0-9]+/roles/[0-9]+/criteria$ 1; ~^/api/v1/microcosms/[0-9]+/roles/[0-9]+/criteria/[0-9]+$ 1; ~^/api/v1/microcosms/[0-9]+/roles/[0-9]+/members$ 1; ~^/api/v1/microcosms/[0-9]+/roles/[0-9]+/profiles$ 1; ~^/api/v1/microcosms/[0-9]+/roles/[0-9]+/profiles/[0-9]+$ 1; ~^/api/v1/microcosms/[0-9]+/roles/[0-9a-zA-Z_-]+$ 1; /api/v1/microcosms/tree 1; ~^/api/v1/out/[2-9a-zA-Z]+$ 1; /api/v1/permission 1; /api/v1/polls 1; ~^/api/v1/polls/[0-9]+$ 1; ~^/api/v1/polls/[0-9]+/attributes$ 1; ~^/api/v1/polls/[0-9]+/attributes/[0-9a-zA-Z_-]+$ 1; ~^/api/v1/polls/[0-9]+/lastcomment$ 1; ~^/api/v1/polls/[0-9]+/newcomment$ 1; /api/v1/profiles 1; ~^/api/v1/profiles/[0-9]+$ 1; ~^/api/v1/profiles/[0-9]+/attachments$ 1; ~^/api/v1/profiles/[0-9]+/attachments/[0-9A-Za-z]+$ 1; ~^/api/v1/profiles/[0-9]+/attachments/[0-9A-Za-z]+.[A-Za-z]+$ 1; ~^/api/v1/profiles/[0-9]+/attributes$ 1; ~^/api/v1/profiles/[0-9]+/attributes/[0-9a-zA-Z_-]+$ 1; /api/v1/profiles/options 1; /api/v1/profiles/read 1; ~^/api/v1/reserved/[0-9a-zA-Z]+$ 1; /api/v1/resolve 1; /api/v1/roles 1; ~^/api/v1/roles/[0-9]+$ 1; ~^/api/v1/roles/[0-9]+/criteria$ 1; ~^/api/v1/roles/[0-9]+/criteria/[0-9]+$ 1; ~^/api/v1/roles/[0-9]+/members$ 1; ~^/api/v1/roles/[0-9]+/profiles$ 1; ~^/api/v1/roles/[0-9]+/profiles/[0-9]+$ 1; /api/v1/search 1; /api/v1/site 1; ~^/api/v1/site/[0-9]+/attributes$ 1; ~^/api/v1/site/[0-9]+/attributes/[0-9a-zA-Z_-]+$ 1; /api/v1/site/menu 1; /api/v1/sites 1; ~^/api/v1/sites/[0-9]+$ 1; ~^/api/v1/sites/[0-9]+/menu$ 1; ~^/api/v1/sites/[0-9]+/status$ 1; /api/v1/trending 1; /api/v1/updates 1; /api/v1/updates/preferences 1; ~^/api/v1/updates/preferences/[0-9]+$ 1; /api/v1/users 1; ~^/api/v1/users/[0-9]+$ 1; /api/v1/users/batch 1; /api/v1/watchers 1; ~^/api/v1/watchers/[0-9]+$ 1; /api/v1/watchers/delete 1; /api/v1/watchers/patch 1; /api/v1/whoami 1; ~^/auth0login/?$ 1; ~^/comments/[0-9]+/?$ 1; ~^/comments/[0-9]+/attachments/?$ 1; ~^/comments/[0-9]+/delete/?$ 1; ~^/comments/[0-9]+/edit/?$ 1; ~^/comments/[0-9]+/incontext/?$ 1; ~^/comments/[0-9]+/source/?$ 1; ~^/comments/create/?$ 1; ~^/compare/?$ 1; ~^/conversations/[0-9]+/?$ 1; ~^/conversations/[0-9]+/delete/?$ 1; ~^/conversations/[0-9]+/edit/?$ 1; ~^/conversations/[0-9]+/newest/?$ 1; ~^/dashboard/?$ 1; ~^/dashboard/sites/?$ 1; ~^/dashboard/sites/create/?$ 1; ~^/dashboard/sites/edit/[0-9]+$ 1; ~^/developers/?$ 1; ~^/error/?$ 1; ~^/events/[0-9]+/?$ 1; ~^/events/[0-9]+/csv$ 1; ~^/events/[0-9]+/delete$ 1; ~^/events/[0-9]+/edit$ 1; ~^/events/[0-9]+/newest$ 1; ~^/events/[0-9]+/rsvp$ 1; ~^/faqs/?$ 1; /favicon.ico 1; ~^/features/?$ 1; ~^/forbidden/?$ 1; ~^/geocode/?$ 1; ~^/headers/?$ 1; ~^/huddles/?$ 1; ~^/huddles/[0-9]+/?$ 1; ~^/huddles/[0-9]+/invite/?$ 1; ~^/huddles/[0-9]+/leave/?$ 1; ~^/huddles/[0-9]+/newest/?$ 1; ~^/huddles/create/?$ 1; ~^/ignore/?$ 1; ~^/ignored/?$ 1; ~^/login/?$ 1; ~^/logout/?$ 1; ~^/microcosms/?$ 1; ~^/microcosms/[0-9]+/?$ 1; ~^/microcosms/[0-9]+/create/conversation/?$ 1; ~^/microcosms/[0-9]+/create/event/?$ 1; ~^/microcosms/[0-9]+/create/microcosm/?$ 1; ~^/microcosms/[0-9]+/delete/?$ 1; ~^/microcosms/[0-9]+/edit/?$ 1; ~^/microcosms/[0-9]+/memberships/?$ 1; ~^/microcosms/[0-9]+/memberships/[0-9]+/api/?$ 1; ~^/microcosms/[0-9]+/memberships/[0-9]+/edit/?$ 1; ~^/microcosms/[0-9]+/memberships/create/?$ 1; ~^/microcosms/create/?$ 1; ~^/moderate/?$ 1; ~^/moderate/do/?$ 1; ~^/notfound/?$ 1; ~^/out/[2-9a-zA-Z]+$ 1; ~^/profiles/?$ 1; ~^/profiles/[0-9]+/?$ 1; ~^/profiles/[0-9]+/edit/?$ 1; ~^/profiles/[0-9]+/patch/?$ 1; ~^/profiles/read/?$ 1; /robots.txt 1; ~^/search/?$ 1; ~^/static/.*$ 1; ~^/terms/?$ 1; ~^/today/?$ 1; ~^/trending/?$ 1; ~^/unignore/?$ 1; ~^/updates/?$ 1; ~^/updates/december/?$ 1; ~^/updates/settings/?$ 1; ~^/watchers/?$ 1; default 0; }
In the
server
part of the site specific config, and as early as possible:# Allow only known URIs if ($known_uri = 0) { return 404; }
And you can try it easily... just access something that isn't in that list, like https://www.lfgss.com/doesnotexist and you get a 404 not found error.
-
• #74
OK, so I got ahead of myself and did have a bug, but hopefully now resolved.
Yup... it fixed it... and now we have a very hardened Nginx config.
-
• #75
monitored for 30 minutes, and only the one false negative detected (redirects on successfully posting a comment were broken)... and now everything is working fine.
I shall go for dinner (it's late evening in Heraklion) and if there are more issues I'll check them later.
I know I already said thanks, but I don't think it was effusive enough. You are a legend for keeping this place going. The rest of the internet fucking sucks now and I've deleted every single account or app I have on every other platform. This place is a living relic of the days when the internet felt like a good and hopeful thing. And its survival is down to you putting in the time to keep it going even though you have what sounds like an incredibly demanding job / being on holiday.