Data still matters and we're running out, from what I can tell.

The easy good data has been collected. Synthetic data is doing its best to fill the gaps. The next big task will be backfilling real-world data that never lived well on the web the way code did.

The other day I was talking with a family member about how good AI has gotten, and it really has, but he was complaining how it wasn't as good for him and people in his orbit of agriculture.

As we talked about it, we started talking about a niche hobby of looking for arrowheads with AI. So we tried it! It did terribly. Now, this was not a rigorous test, but to me it showed how challenging the data gap is for information that never lived naturally on the web for training.

The next frontier of AI seems to be specialized hardware and specialized data. For now, people will collect and curate this data, but it strikes me that the next generation of mechanisms will need to allow for AIs to source their own data and build tooling to discover and learn this information.