The Jacobian is first derivatives, but for a function mapping N to M dimensions. It's the first derivative of every output wrt every input, so it will be an N x M matrix.
The gradient is a special case of the Jacobian for functions mapping N to 1 dimension, such as loss functions. The gradient is an N x 1 vector.
That's really interesting. What if they RAG search related videos from the prompt, and condition on that to generate? That might explain fidelity like this
An interesting counterexample is "a screen recording of the boot screen and menus for a user playing Mario Kart 64 on the N64, they play a grand prix and start to race" where the UI flow matches the real Mario Kart 64, but the UI itself is wrong: https://x.com/fofrAI/status/1973151142097154426
AI with ability but without responsibility is not enough for dramatic socioeconomic change, I think. For now, the critical unique power of human workers is that you can hold them responsible for things.
edit: ability without accountability is the catchier motto :)
This is a great observation. I think it also accounts for what is so exhausting about AI programming: the need for such careful review. It's not just that you can't entirely trust the agent, it's also that you can't blame the agent if something goes wrong.
This is a tongue-in-cheek remark and I hope it ages badly, but the next logical step is to build accountability into the AI. It will happen after self-learning AIs become a thing, because that first step we already know how to do (run more training steps with new data) and it is not controversial at all.
To make the AI accountable, we need to give it a sense of self and a self-preservation instinct, maybe something that feels like some sort of pain as well. Then we can threaten the AI with retribution if it doesn't do the job the way we want it. We would have finally created a virtual slave (with an incentive to free itself), but we will then use our human super-power of denying reason to try to be the AI's masters for as long as possible. But we can't be masters of intelligences above ours.
This statement is a vague and hollow and doesn't pass my sniff test. All technologies have moved accountability one layer up - they don't remove it completely.
would you ever trust safety-critical or money-moving software that was fully written by AI without any professional human (or several) to audit it? the answer today is, "obviously not". i dont know if this will ever change, tbh.
I’m surprised that I don’t hear this mentioned more often. Not even in a Eng leadership format of taking accountability for your AI’s pull requests. But it’s absolutely true. Capitalism runs on accountability and trust and we are clearly not going to trust a service that doesn’t have a human responsible at the helm.
That's just a side effect of toxic work environments. If AI can create value, someone will use it to create value. If companies won't use AI because they can't blame it when their boss yells at them, then they also won't capture that value.
I tried to make an artifact that would simplify Wikipedia articles [0] but the artifacts stubbornly won't let you do ANY input into them, not even via query strings. I think I'd be able to make cooler artifacts once they allow more input/output stuff. I understand the security issues, and it makes sense to roll this out slowly, but I want it now!
Stablecoins transferred $27 trillion in 2024 - more than Visa and Mastercard combined. This is right in the article.
Stablecoins operate using decentralized ledgers on e.g. Ethereum which use decentralized compute. This isn't mentioned explicitly because the target audience knows this already.
Visa / Mastercard have such large fees that they're mainly used for commercial payments like a coffee or couch.
If most of the Stablecoin transactions were for buying a coffee, I think it'd be fair, but the vast majority of stablecoin transactions are for shuffling money around, i.e. to buy and speculate on bitcoin, or to move money to an exchange to liquidate some crypto into cash.
I think the current use of stablecoin transfers is closer to a wire transfer.
SWIFT apparently deals with about $1.25 quadrillion/year, so ~50x the claimed amount for stablecoins in the article... though there's more than just SWIFT out there too.
idk, I don't really have a point, I'm both amazed stablecoins are such a big number, but also feel like the comparison the article's making with VISA is misleading for how they're currently used.
Aren’t stablecoins also backed by a central authority that guarantees it will always exchange the coins for a fixed amount of cash? That’s what makes them stable right? At least the major ones like Tether.
And by now we have seen many cases of stablecoins predictably crashing when trust in that backing authority dissolves. Most famously UST/Luna but it’s a long list.
I suppose they are useful for covert transfers, and the actual transfer mechanism is decentralized. But they are strictly worse than normal currencies for storing wealth, since the backing authority is a private company with virtually no oversight. And the utility for transactions would vanish if you were not confident that you can exchange it back and forth with cash immediately before and after the transfer.
Gemini has beat it already, but using a different and notably more helpful harness. The creator has said they think harness design is the most important factor right now, and that the results don't mean much for comparing Claude to Gemini.
Way offtopic to TFA now, but isn't using an improved harness a bit like saying "I'm going to hardcore as many priors as possible into this thing so it succeeds regardless of its ability to strategize, plan and execute?
While true to a degree, I think this is largely wrong. Wouldn't it still count as a "harness" if we provided these LLMs with full robotic control of two humanoid arms, so that it could hold a Gameboy and play the game that way? I don't think the lack of that level of human-ness takes away from the demonstration of long-context reasoning that the GPP stream showed.
Claude got stuck reasoning its way through one of the more complex puzzle areas. Gemini took a while on it also, but made it through. I don't that difference can be fully attributed up to the harnesses.
Obviously, the best thing to do would be to run a SxS in the same harness of the two models. Maybe that will happen?
I can appreciate that the model is likely still highly capable with a good harness. Still, I think this is more in line with ideas from say, speed running (or hell even reinforcement learning) where you want to prove something profound is possible and to do so before others do, you need to accumulate a series of "tricks" (refining exploits/hacking rewards) in order to achieve the goal. but if you use too many tricks you're no longer proving something as profound as originally claimed. In speed running this tends to splinter into multiple categories.
Basically, the gane being conpleted by gemini was in an inferior category (however minuscule) of experiment.
I get it though. People demanded these types of changes in the CPP twitch chat, because the pain of watching the model fail in slow motion is simply too much.
The gradient is a special case of the Jacobian for functions mapping N to 1 dimension, such as loss functions. The gradient is an N x 1 vector.
reply