- Your objection, as I understand it, is some combination of "no true Scotsman" combined with complaints about wide events themselves.
To the first point (no true Scotsman), I really don't think that applies in the slightest. The post I'm replying to said (paraphrasing) that middle-layer observability is hard with wide events and easy with logs. My counter is that the objection has nothing to do with wide events vs logs, since in both scenarios you can choose to include or omit more information with the same syntactic (and similar runtime overhead) ease. I think that's qualitatively different from other NTS arguments like TDD, in that their complaint is "I don't have enough information if I don't send it somewhere" and my objection is just "have you tried sending it somewhere?" There isn't an infinite stream of counter-arguments about holding the tool wrong; there's the very dumbest suggestion a rubber duck might possibly provide about their particular complaint, which is fully and easily compatible with wide events in every incarnation I've seen.
To the second point (wide events aren't especially useful and/or suck), I think a proper argument for them is a bit more nuanced (and I agree that they aren't a panacea and aren't without their drawbacks). I'll devote the rest of my (hopefully brief) comment to this idea.
1. Your counter-example falls prey to the same flaw as the post I responded to. If you want information then just send that information somewhere. Wide events don't stop you from gathering data you care about. If you need a requestID then it likely exists in the event already. If you need a message then _hopefully_ that's reasonably encoded in your choice of sub-field, and if it's not then you're free to tack on that sort of metadata as well.
2. Your next objection is about the wide event being so context-rich as to be a problem in its own right. I'm sympathetic to that issue, but normal logging doesn't avoid it. It takes exactly one production issue where you can't tie together related events (or else can tie them together but only via hacks which sometimes merge unrelated events with similar strings) for you to realize that completely disjoint log lines aren't exactly a safe fallback. If you have so much context-dependent complexity that a wide event is hard to interpret then linear logs are going to be a pain in the ass as well.
Mildly addressing the _actual_ pros and cons: Logs and wide events are both capapable of transmitting the same information. One reasonable frame of reference is viewing wide events as "pre-joined" with a side helping of compiler enforcement of the structure.
It's trivially possible to produce two log lines in unrelated parts of a code base which no possible parser can disambiguate. That's not (usually) possible when you have some data specification (your wide event) mediating the madness.
It's sometimes possible with normal logs to join on things which matter (as in your requestID example), but it's always possible with wide events since the relevant joins are executed by construction (also substantially more cheaply than a post-hoc join). Moreover, when you have to sub-sample, wide events give an easy strategy for ensuring your joins work (you sub-sample at a wide event level rather than a log-line level) -- it's not required; I've worked on systems with a "log seed" or whatever which manage that joinability in the face of sub-sampling, but it's more likely to "just work" with wide events.
The real argument in favor of wide events, IMO, is that it encourages returning information a caller is likely to care about at every level of the stack. You don't just get potentially slightly better logs; you're able to leverage the information in better tests and other hooks into the system in question. Parsing logs for every little one-off task sucks, and systems designed to be treated that way tend to suck and be nearly impossible to modify as desired if you actually have to interact with "logs" programatically.
It's still just one design choice in a sea of other tradeoffs (and one I'm only half-heartledly pursuing at $WORK since we definitely have some constraints which are solved by neither wide events nor traditional logging), BUT:
1. My argument against some random person's choice of counter-argument is perfectly sound. Nothing they said depended on wide events in the slightest, which was my core complaint, and I'm very mildly offended that anyone capable of writing something as otherwise sane and structured as your response would think otherwise.
2. Wide events do have a purpose, and your response doesn't seem to recognize that point in the design space. TFA wasn't the most enjoyable thing I've ever read, but I don't think the core ideas were that opaque, and I don't think a moment's consideration of carry-on implications would be out of the question either. I could be very wrong about the requisite background to understand the article or something, but I'm surprised to see responses of any nature which engage with irrelevant minutea rather than the subset of core benefits TFA chose to highlight (and I'm even more surprised to see anything in favor of or against wide events given my stated position that I care more about the faulty argument against them than whether they're good or bad)..
- If that's an issue (visibility into middle layers) it just means your events aren't wide enough. There's no fundamental difference between log.error(data) and wide_event.attach(error, data), nor similar schemes using parameters rather that context/global-based state.
There are still use cases for one or the other strategy, but I'm not a fan of this explanation in either direction.
- I largely agree, but there's a little nuance insofar as "interior-point" methods are very powerful. You can go a long way by encoding your goals as error functions and letting a gradient-based optimizer do the rest.
- I've done that for desktop apps before. You have to be careful with the effects of sub-pixel rendering and whatnot if your math is continuous, but it's a viable path that I quite like.
- To be fair, this particular issue wouldn't have happened in C, Python, Forth, Zig, or a host of other languages. String-based weirdness is something of a JS issue.
- That behavior is what finally got me off Facebook awhile back.
Edit: And something similar with Windows now that I think about it; there was a privacy setting which would appear to work till you re-entered that menu. Saving the setting didn't actually persist it, and the default was not consumer-friendly.
- This is one of those places where it's worth disentangling the status quo from an optimal alternative.
Currently, every factlet you leak to one of these systems poisons them toward their profit (and almost unanimously against your best interests). Advertising draws your attention away from the products that would make your life better (cheaper, heathier, tastier, whatever) and toward profitable alternatives.
It doesn't have to be that way though. You physically don't have time to research every thing that exists, or even to hear about every possible product in passing. Supposing some of those would improve your life on average, is word-of-mouth really the most efficient way we can come up with to tell you about the things you do actually want to spend your money on? In theory, this is a great business -- customers want to spend money, companies want to sell things, and the information/discoverability asymmetry means that companies are inclined to get word of their products out there with customers _also_ wanting to hear about those products (if they're sufficiently personalized). If "advertising" were good enough, I'd pay money for it.
That only falls apart because of a lack of trust and ethical behavior. Instead of being treated like the information market it is, it's thrust onto individuals to try to prey on their weaknesses.
- I haven't heard.
- That was less my point, and more that "battle-tested" doesn't have to be a cudgel to argue against in-house projects, especially when considering defect rates (the more-general solution is very often slower and buggier to support the features you don't need).
- My understanding is that the iouring CVEs are about local privilege escalation, not being appropriately sandboxed, etc. If you're only running code you trust on machines with iouring enabled then you're fine (give or take "defense in depth").
Is that not accurate?
- Nice. I was looking at building an object store myself. It's fun to see what features other people think are important.
I'm curious about one aspect though. The price comparison says storage is "included," but that hides the fact that you only have 2TB on the suggested instance type, bringing the storage cost to $180/TB/mo if you pay each year up-front for savings, $540/TB/mo when you consider that the durability solution is vanilla replication.
I know that's "double counting" or whatever, but the read/write workloads being suggested here are strange to me. If you only have 1875GB of data (achieved with 3 of those instances because of replication) and sustain 10k small-object (4KiB) QPS as per the other part of the cost comparison, you're describing a world where you read and/or write 50x your entire storage capacity every month.
I know there can be hot vs cold objects or workloads where most data is transient, but even then that still feels like a lot higher access amplification than I would expect from most workloads (or have ever observed in any job I'm allowed to write about publicly). With that in mind, the storage costs themselves actually dominate, and you're at the mercy of AWS not providing any solution even as cheap as 6x the cost of a 2-year amortized SSD (and only S3 comes close -- it's worse when you rent actual "disks," doubly so when they're high-performance).
- I think their point is that system timestamps for that append-only format aren't good enough. You need logical timestamps corresponding to increasing transaction ids.
- You mean this log4j [0] with major vulnerabilities the industry missed for nearly a decade?
- That's Uber's approach, right?
- And you've spotted why normal, regular, simple, natural, real mathematics is so hard for people to grok at first glance.
- That brought out a nice, hearty chuckle. At least you're owning your style.
- > number of chars minimum not the goal
For my tastes this is excessive (especially since the formatting is non-existent), but you do pay a cost for longer variable names. There's only so much you can grok when reading a passage, and long names hinder any higher-level understanding of what that passage actually does, requiring you to resort to carefully thinking about the problem rather than any shortcuts and insights your powerful visual cortex might provide.
- That's... intriguing. I just tested out functiontrace and saw 20-30% overhead. I didn't expect it to be anywhere near that low.
- That depends on the code you're profiling. Even good line profilers can add 2-5x overhead on programs not optimized for them, and you're in a bit of a pickle because the programs least optimized for line profiling are those which are already "optimized" (fast results for a given task when written in Python).
It's really not just that though. A lot goes into it, but one observation is that the relative increases in wages and prices isn't distributed evenly. Some examples:
- A lot of people are legitimately substantially better off than they would have been a few decades ago. I literally never have to worry about money anymore when thinking about our purchases (for everything but a house with a big yard, which we still can't safely [0] afford without moving). I'm not alone.
- That's not true of everyone, even my next-door neighbors. I know people splitting a studio apartment and still struggling a bit. They have good jobs, and even splitting the apartment their post-tax, post-rent pay is $7.20/hr. That's fine enough I suppose, but they'll literally never be able to save for a home of any quality in the area in their entire lives using only a single income. It'll take them awhile to afford a home anywhere.
- Suppose you have a couple young kids. That places hard bounds on how much money you need to make even for childcare to make sense to get up to two incomes in the first place. I've known plenty of people with PhDs and good jobs who quit to take care of the kids for financial reasons, supporting the household on just the higher-earner's pay.
- A lot of small towns haven't seen the same increase in wages as the rest of the country but have seen the increase in prices. My hometown saw an increase from $10/hr to $20/hr in what a great wage is over the last 25 years. CPI only went up 1.9x in that time, but the same caliber of house went up 3x, and the staples people used to eat (like ground beef) went up more than 3x as well. They're correctly observing that they have less take-home money (because of 3x increased rent), that take-home money doesn't go as far (they can't eat the same foods they could 25yrs ago), and it definitely doesn't go as far if you want to do something like save for a house (it's an extra 4+yrs of post-tax, post-rent income to pay for a house, assuming you could devote all of it to savings instead of groceries and whatnot).
I'm not sure exactly how to quantify who's struggling and why at a macroscopic level, but I guarantee they're real and that it's not just an increase in expectations.
[0] It depends on your relative risk levels, but if you're not convinced the gravy train will last forever and are concerned about locking up all your assets in a depreciating vehicle then you need to be a bit more frugle with your choice of home.