5 Comments

In regards to this line: “GPT-4 should be thought of more like a large-scale oil refinery operated by one of the ancient vast oil corporations at the dawn of the oil era” — I wrote up a piece forecasting authoritarian and sovereign power uses of LLMs (https://taboo.substack.com/p/authoritarian-large-language-models). One thing I'm curious about is how many countries will start treating access to LLMs as a national security economic risk in the same way we treat access to oil. How often LLMs will be used in embargoes? If a country or company controls a powerful one, they could cut off API access.

Expand full comment

* "capability overhang" great term! The political angle is well said, and needs to be repeated by folks with following. I covered the same thing in my piece in the societal section, and called it the "Keynote Era" of ML, equating them to tech companies rather than oiligopolies https://robotic.substack.com/

* RE Alpaca, we've been seeing it's less good on factual answers, while being decent at this "fun stuff". I wonder if that's a result of being fine-tuned on instruction data with a less good base model.

Expand full comment