Hard times
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    I got march of shamed once because I genuinely forgot, it somehow shortcircuits my brain when I need to wait for approval, thinking I've already paid and walk out after waiting for the person to come tap some buttons :|

    9
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 100%
    Alternative contacts for admins

    Since many people have asked me already, I created a Mastodon here: https://mastodon.social/@latteisnotcoffee You can follow that for server downtime, updates when this server is down etc :)

    2
    0
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta 1y ago
    Jump
    Server has been successfully moved
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    No need for now, if it gets to be more of a handful I will start asking for mods, maybe crowdfund a larger server etc but I suspect it'll be alright :)

    1
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta 1y ago
    Jump
    Server has been successfully moved
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 0%

    Aye we're still on 0.17.4, mostly because I was lazy, but it seems to have worked out in this case. I'll give 0.18 a little time to settle before we upgrade

    0
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 100%
    Server has been successfully moved

    Sorry for the downtime, the server has been moved onto better infrastructure so hopefully it won't go down as frequently in the future

    3
    8
    Weekly /c/china discussion post
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    That sounds like a particularly horrible idea, there's no way to know. I would take the risk on electronics, not something you actually put into your body.

    1
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 66%
    0.18 released, should we upgrade?

    They killed captcha support in 0.18, which basically negates one of the methods to stop the bots... so if we upgrade, we will likely need to email verification until 0.18.1 is released OR stay on 0.17.4 until 0.18.1 is released where they've promised to bring it back.

    1
    7
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Jump
    Is there an easy way to know who is spamming from my instance and contact other admins?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    Not super easily. It can be done by querying the postgresql dabase, but there is no built-in method to do it using the browser interface at the moment. When anyone from any instance does report them, you will see the report.

    Do you have an example of a query I could use? I'm a bit rusty with databases :) with an example I can get chatgpt to build me something better

    2
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Jump
    Is there an easy way to know who is spamming from my instance and contact other admins?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    I see, occasionally I get reports from local users about something elsewhere from a 'foreign' user... which I can then delete. I assume that's not deleting from their server?

    For example I get a report from a local user about a lemmy.ml user with a post in lemmy.world

    2
  • Can you help me deal with the bots?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    I don't unfortunately, I deliberately don't log that due to some of the sensitive stuff on my own instance (we're China based)

    2
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support coffeeisnotlatte 1y ago 100%
    Is there an easy way to know who is spamming from my instance and contact other admins?

    After the bot attacks I've cleaned up and implemented some additional measures thanks to some helpful users, but I'd also like to monitor the situation in case someone from my server decides to go on a spam rampage... Is there a way to: 1) Easily get a list of comments from all my users locally and across the 'verse? 2) If I purge/ban these users, is there a protocol for letting other Lemmy admins know? I assume I have no power to delete things from their instances (just my copy of their instance?)

    8
    6
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 100%
    Registration is now reopened

    Bots cleaned, there is now a captcha and I have some additional admin tools. I'm not going into too much detail as the bot maker may be reading :) I will be keeping a close eye on the numbers and patterns over the next few days, so it may close again for a few hours depending on the types of attacks. If anyone sees a latte.isnot.coffee user spamming anywhere on Lemmy please let me know immediately

    1
    0
    Can you help me deal with the bots?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    Yup I've got them, luckily 5 or more repeating numbers in their email pretty much identified 99% of them. Would you like me to send the CSV somewhere? 27k+ bots

    1
  • Can you help me deal with the bots?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    That's okay I found it, luckily there's a pattern here too... I dug up 27k with repeating numbers on emails which is a good start!

    3
  • Can you help me deal with the bots?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    Huh adminer would definitely be an easier way to do this, do you have the part of the docker-compose you used with the env vars etc?

    2
  • Can you help me deal with the bots?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    Yeah that's the issue I'm having, someone sent me a postgres command in DM earlier but it does seem to be a bit of a nuke/picking up ordinary users....

    3
  • Trying to purge them and set up with Lemmy Ansible, I've disabled signups without admin approval for now. I see a postgres container and a password but I'm not very familiar with postgres, I tried psql but can't get access Edit: Also anyone who's de-federated with us, please reconsider. We're a small server with active admin and will get a handle on this, I promise! We had an easy process to sign up for a few days while I got my users over and forgot to turn it off Edit2: Looking much more healthy now, I will put the commands I've used in a comment below

    14
    18
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 100%
    Admin approval now reqd for signup (Temporary)

    Admin approval will be granted every few hours, not ideal but I need to get a handle on the 28k+ bots that have signed up.

    1
    0
    Self Hosted Lemmy instance comments terribly out of sync?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    Yeah I started on a dinky little server and now it's massive.... mostly seems to be a memleak or something going on though that's causing OOM and a massive CPU spike. Anyone know anything? 4GB of RAM getting eaten for breakfast... just had to up it to 8

    1
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support coffeeisnotlatte 1y ago 100%
    We seem to be memleaking

    On 0.17.4, has anyone else noticed this? The last few days the server chewed through 2GB of RAM... then yesterday it chewed through 4GB after an upgrade in about 2 days and now seems to be chewing through about 100MB/hour with no sign of stopping. I put in a bug report but thought I'd ask if anyone else is having the issue too: https://github.com/LemmyNet/lemmy/issues/3183

    4
    0
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 100%
    Downtime again + cause seems to be a memleak

    Seems we have a memory leak in Lemmy, I will report it upstream and see what they say since we have 4GB RAM now and it was fine with 2GB for days and now goes down with 4 over 24 hours :| All I will say is bloody node.js > ./syslog:Jun 18 04:38:38 ip-172-31-33-220 kernel: [142633.319174] Out of memory: Killed process 1349 (node) total-vm:2700484kB, anon-rss:2004100kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:18016kB oom_score_adj:0 Reported upstream, in the meantime I may need to do a quick reboot every 24 hours where the server may be down for 1-2 minutes: https://github.com/LemmyNet/lemmy/issues/3183

    1
    0
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 100%
    Sorry for downtime

    Server went crazy in CPU and killed itself, looking into it now.

    1
    0
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support coffeeisnotlatte 1y ago 100%
    Why am I getting reports from other instances?

    I'm not on either the instance reported from or the instance reported to, I have my own instance. Why am I getting these reports/would care about them?

    5
    0
    What are YOU self-hosting?
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    Mine is actually also a VPS, my electricity is too unstable to host at home :')

    It's pretty light though, 2GB RAM and 2vCPUs and about 10GB of space. It shot up to 9.5GB over a few days as people added more communities, then slowed down a lot. We have ~60 users as of now

    2
  • All the sad
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCO
    coffeeisnotlatte
    1y ago 100%

    Regular Apollo still works for now, I've got the modified version on a backup iPhone 8 so let's see if it still works there after the shutdown

    4
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearME
    meta coffeeisnotlatte 1y ago 100%
    kitten
    2
    0
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearCA
    cats coffeeisnotlatte 1y ago 100%
    Cat
    1
    0

    This is a general discussion thread for any questions or topics that you feel don't deserve their own thread, or just for random thoughts and comments. The sidebar guidelines apply here too and these threads will be closely moderated, so please keep the discussions civil, and try to keep top-level comments China-related.

    1
    3
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support coffeeisnotlatte 1y ago 100%
    Comments aren't loading for other instances?

    As you can see in the screenshot, most communities I put in yesterday (except this one) have zero comments. Any idea why that would be? If I go to the instance itself I see comments

    3
    2
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support coffeeisnotlatte 1y ago 100%
    Testing

    Testing 123

    1
    0