ToolsLib services status

  1. 2016-09-27 14:00:00
    We are migrating one of our hypervisor: ECC RAM, more I/Os, more stable network… It will have an impact on several VMs, including the mail one. However, we shouldn’t lose any email, at most ehy may be delayed a few minutes.
  2. 2016-09-07 22:00:00
    We patch the upstream fix for CVE-2016-5696 on our concerned servers. More informations: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-5696 This requires a reboot for our servers. No downtime expected on ToolsLib, but around a minute is expected on other services.
  3. 2016-08-31 21:20:00
    The database cluster will be upgraded. ~5 seconds of downtime at most is expected. The slave crashed during the upgrade, so we rebooted it (without impact on the service). The master is currently freezed, we are rebooting it too… It causes a downtime on our different services. Everything is up. ~90 secondes of downtime observed.
  4. 2016-08-04 12:15:00
    We are upgrading our development application. One minute of downtime is expected. Everything is up & running.
  5. 2016-07-26 04:45:00
    We are upgrading our development application. One minute of downtime is expected. Everything is up and running.
  6. 2016-07-20 19:30:00
    We are upgrading our cloud application. Around one minute of downtime is expected. Everything is up and running.
  7. 2016-07-18 12:25:00
    We are upgrading our development application. One minute of downtime is expected. Everything is up and running.
  8. 2016-07-03 14:30:00
    We are upgrading our cloud application. Around one minute of downtime is expected. Everything is now up and running.
  9. 2016-06-25 13:20:00
    We are upgrading our pad application. A few minutes of downtime is expected. Everything is now up and running.
  10. 2016-06-07 11:30:00
    A router was causing troubles in one of our hosting provider. They rebooted it, and it has caused a traffic downtime on part of our infrastructure. In the meantime, the redondant part was still online and reachable, so there was no service disruption. Around 11:55, everything was fixed. Below a copy of our hosting provider ticket : We are facing an issue on this router causing DHCPACK packets to get blocked and thus preventing servers from rebooting correctly.
  11. 2016-05-24 18:30:00
    We are upgrading our webmail application. A few seconds of downtime is expected, but the IMAP/POP protocols are still working. Everything is up. We have also tweaked the configuration settings.
  12. 2016-05-18 18:58:00
    We are currently upgrading the database server. Several minutes of downtime should be expected, but hopefully without dataloss. The new server is now live and running well.
  13. 2016-05-18 18:16:00
    The database cluster will be upgraded. ~5 seconds of downtime at most is expected. Everything is up and running.
  14. 2016-05-08 14:30:00
    We are upgrading our cloud application. A few seconds of downtime is expected. Everything is up and running.
  15. 2016-05-05 14:30:00
    We are upgrading our webmail application. A few seconds of downtime is expected, but the IMAP/POP protocols are still working. Everything is now up and running.
  16. 2016-04-25 12:00:00
    We are upgrading our pad application. A few minutes of downtime is expected. Everything is up and running.
  17. 2016-04-07 20:00:00
    We are upgrading our cloud application. A few seconds of downtime is expected. Everything is up and running.
  18. 2016-04-07 15:30:00
    Some recipients appear to receive some emails from ToolsLib in their junk folder. We contacted Microsoft about the issue, they confirmed our configuration was not in cause. The Smartscreen filter seems too strict, but they told us to wait for the issue to be fixed. A workaround is to sign every emails with GPG, which make them pass the Smartscreen filter.
  19. 2016-04-05 12:51:00
    pnmail.nl refuses to talk to one of the mail server we used due to a blacklist by Symantec. We contacted them to resolve the issue, it’s pending for approval and should be fixed in a few hours.
  20. 2016-04-03 20:00:00
    We are upgrading one of our hypervisor. It will impact mail, cloud, and dev platforms. 04/04/16 - 21:00:00 (UTC+2) : We faced some issues which delayed the migration, but the initial vms are operational now. We are currently migrating another hypervisor which should led for a small downtime of https://up2sha.re, our dns server and our storage of visitors statistics. 05/04/16 - 04:00:00 (UTC+2) : Nearly all vms are migrated. There may be some issues with IPV6 connectivity for DNS and Up2Sha.
  21. 2016-03-30 20:28:00
    Following the migration to the new infrastructure , it appears that some part of it were not performant enough to manage all the increasing traffic. We adapted them consequently and there is a huge benefits in performances. This migration allows us to deliver 2.*1 Gbps in throughput. No downtime has been observed. We’ll write a detailled blogpost in the coming days.
  22. 2016-03-27 20:28:00
    Following the database cluster migration, we’ll progressively move the whole platform. It will be progressive and should not cause any traffic interruption. This migration allows us to be more resilient and to scale efficiently. The migration is progressively done (the time for the clients to update their DNS record) and we’re watching the behaviour of the new infrastructure. A hotfix has been made to fix the delivery of assets in HTTPS, the complete fix will come in the next minutes/hours.
  23. 2016-02-09 08:45:00
    The whole platform is unreachable for a few minutes. We’re investigating and trying to recover asap. A kernel crash is in cause. All services are up and we’re setting a plan to avoid this problem again.
  24. 2016-01-29 20:16:00
    The database cluster will be upgraded. ~5 seconds of downtime at most is expected. Everything is now fully functional. 3 seconds of downtime observed.
  25. 2016-01-19 22:49:00
    20/01/2016 1:56 : Everything is up and running. 20/01/2016 01:40 : Patch currently applied on the last server. A small downtime expected. 20/01/2016 00:30 : Patch applied nearly everywhere, one server remains. Everything is up, the impact was minimal. We patch the CVE-2016-0728 on our concerned servers. More informations : https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2016-0728
  26. 2015-12-28 20:28:00
    We are upgrading the databases backend to MariaDB 10.1.10. A few seconds of downtime is expected. We’ll begin by the slaves, and continue with masters if everything goes well. Everything has been updated and run smoothy. Less than 10 seconds of downtime have been observed.
  27. 2015-12-28 20:00:00
    We are upgrading our cloud application. A few seconds of downtime is expected. Everything is now operational.
  28. 2015-12-14 20:58:00
    The backup server has freezed, probably due to a high temperature : Fans seems off, we are waiting an intervention from our hosting provider to fix them. There is no impact on public services, nor data stored on this server. The cooling system has been changed, the CPU is cooler than before, we’ll watch it carefully the coming days.
  29. 2015-12-13 00:01:00
    A few mail accounts don’t receive incoming emails. The problem has been identified and fixed. It can take a couple of hours to be fully efficient. Everything is now functional.
  30. 2015-11-18 19:14:59
    We just upgraded the platform. Nearly two minutes of downtime, everything is up and up-to-date now.
  31. 2015-11-18 12:38:00
    The Linux kernel of the Up2sha.re machine has crashed. We’re investigating for the causes. The service is up after a reboot. An outage occured on Up2sha.re. It has been identified but the sysadmin team was unable to act quickly enough. The cause was the system clock was shifted and it the machine hanged during the backup process. The problem has been temporarly fixed, and we’ll apply the defintive fixe sunday evening.
  32. 2015-11-15 16:46:00
    We are improving the DNS resolvers infrastructure. No downtime expected. Almost half of the infrastructure is done. Everything goes smoothy ! All done.
  33. 2015-11-08 13:08:00
    We are optimizing the DNS slaves. They should answer faster than before. Maintenance done. Everything is up and no downtime has been observed.
  34. 2015-11-08 12:29:00
    Earthlink.net refuses to talk to one of the mail server we used. We contacted them to resolve the issue. We have removed the block on mail originating from 5.196.175.180, please allow 12-24 hours for normal email traffic to resume.
  35. 2015-11-08 02:00:00
    We are optimizing part of the databases cluster. A few seconds of downtime might be expected for the whole platform. Maintenance done. Everything is up and no downtime has been observed.
  36. 2015-11-07 01:00:00
    A flood of spam is happening since a few minutes. We’re investigating and mitigating it. Newly created user accounts are filtered and malicious one are blocked. The spam flood has ended.