How is the abandoned jobs measurement to work as it’s not clear what percentage of jobs have failed?

New problems require new tasks, we shouldn’t be reopening tasks for two unrelated problems

We could store and average the backend response times variable that we post in web requests - this would be a form of inexpensive latency monitoring

To be honest, I'm thinking we should use sre-infrastructure more broadly, for Infrastructure matters so I don't see a problem really with add you to this group as you have the same access and responsibilities as anyone in sre-infra, just not the primary focus. Anything relevant for them would be relevant for you too.

Unauthorized personally identifying information should be reported to government, they have pages that specifically handle these stuffs as I said above. I don't think Bilibili do have enough privilege to delete user data without their consent unless it's government issued.

We had this conversation on setup and when doing initial testing, we decided to make use of IPv6 to prevent needing to recreate a network topology we couldn’t maintain due to cost and limited power access.

Can we create a non-email based sre-roots for this please?

In T9712#196202, @RhinosF1 wrote:

In T9712#196193, @John wrote:

"Was force pushed without waiting for CI by John @ 19:56"

It wasn't forced pushed - it was committed?

Clarification: CI there to point out exactly this was bypassed by merging without a PR and then when post merged failed completely ignored for about 10 minutes

@Agent_Isai Can this be announced for tomorrow 2000-2030 please?

https://github.com/miraheze/mw-config/commit/212136fee36e73f4c6d8fe1dd022a8af211dedab is where behaviour changes and likely the source of the breaking change

"Was force pushed without waiting for CI by John @ 19:56"

Changes have been applied, but all db servers need a reboot. Will need to schedule some time in for this.

(-Infra as tasks shouldn’t be dual tagged for responsibility)

• DNS
  • SLO: Latency for DNS lookups to be below 5ms at least 99.5%

This feels like am infinitely defined task - where does the scope end? What's the end goal? I think this should be split up into tasks for each part so they can be tracked, fleshed out and opinions can be sought in a more defined manner.

Just for the record, this was informally already approved as was communicated on IRC a few weeks ago :)

As best person is position to do this.

syslog suggests the networking service on cloud10 was restarted

No objections from me

Approved as a transitionary measure

@Paladox Is this something for us to do with switching servers?

-> EM Review to take into account both SRE and T&S review

• Mail
  • SLO: Availability of Mail servers to be at least 99.5%.
  • SLO: Errors for sending mail is below 1%.
  • SLO: Latency of message delivery is below 30 seconds.
• MariaDB
  • SLO: Availability of MariaDB is at least 99.5%.
  • SLO: Error rates for access are below 5%.
• LDAP
  • SLO: Availability of LDAP to be at least 99.5%.

Proposal for some below are:

It’s very likely related, though I’ll note I’ve never experienced this problem

In T9425#191241, @Paladox wrote:

Because there's no MTRs showing the problem or any other evidence of this issue, closing as declined as it is untraceable.