There are lots of ongoing efforts trying to put governance around data. The Australian Privacy Act, GDPR and initiatives like the Farm Data Code are all attempts to create boundaries around how data is collected, stored and used. Privacy itself is an extraordinarily difficult problem and I’m not convinced we will see a complete solution to it in my lifetime, if ever.

At the same time, AI systems have created an insatiable demand for data. Data used to train models that shape the services, interfaces and decisions surrounding us has never been more valuable. The systems we interact with are also extractive, they have been for years but they are also becoming even more so. They do not just collect information about us. The data is used to train models that are then used to infer things about us, classify us, predict us and force us to act on those predictions, rarely optimised for our own best interests and I think in many ways, the horse has bolted on that one.

This week I was reminded of a phrase I had not heard in a while: data dignity. It came up in a podcast conversation between Jaron Lanier and Neil deGrasse Tyson alongside a question that has been bouncing around my head.

What if we focused less on governing the data itself and more on governing the application of models? The next evolution of digital rights is not just the ability to opt out of data collection, but the ability to opt out of model behaviour.

Rather than fighting Facebook, Google, Microsoft etc over whether they can collect my data to train models, perhaps I should have the right to say: do not apply models to me. Do not use emotional prediction systems on my feed. Do not use inferred characteristics to shape pricing, recommendations or opportunities presented to me.

I think that is interesting. Most existing privacy frameworks focus on the possession of data. Increasingly though, the downstream application of models has more significance than the upstream collection itself. Harm does not only emerge from someone having data about me. It can emerge from what systems infer, predict and optimise using that data.

What does an inference breach look like? Is that even a thing?

I do not think this replaces the need for privacy regulation, it augments it, as I said privacy is enormously complex and is quantum like in as much as it changes depending on who’s looking at it. Data governance still matters enormously. But I do wonder whether we are not going to get ahead of the other half of the problem if we stop at collection and consent while thinking that will also cover the behaviour of the systems built on top of that data.

I also suspect this becomes harder as models become more embedded into everyday infrastructure. I feel like as a someone who has been involved in tech for the last 30 years that we didn’t fight hard enough the first time around. Recommendation systems, insurance pricing, lending decisions, advertising, search results and digital assistants are all starting to blur together into one continuous layer of machine judgement. Maybe this is the way machines enslave humanity, by capturing the human mind one model, one interaction at a time, taking away our agency and free will.

The future question has to be not just "who owns my data?" but also "which models are allowed to act on me?"