I love how Bret Victor outlined the importance of programming transforming into a paradigm of direct data manipulation. I’m much more curious about a programming paradigm that no longer uses text to communicate with computers but instead just directly manipulating data, receiving past, present, and future feedback of how it would change given your manipulations. Or to put it a different way “What if” feedback. “If you did this, the data would change in this way” is visualized across many different dimensions, allowing you to ‘feel your way’ through feedback where you wish to go. In other words, you give your computer your input data, and you modify dimensions which allow you to specify what you want the program to do. To be clear, I’m not searching for specialized interpretations of this “Oh someone did this with typography” or “Oh someone did this with a game” but rather some more generalizable form like “Someone tried to replace Python with an idea like this” I suppose the nearest thing I can think of is manually modifying the parameters of a neural net but that’s perhaps too cumbersome because there are so many. Perhaps if you can put an autoencoder on top of that, and reduce the parameters down to a smaller “meta” set of parameters that you can manipulate which manipulate the population of parameters in the larger neural net? I’m just really curious if there have been instantiations along these lines (as opposed to code live-running with results on the sides). I realize this is all quite difficult, may even seem ‘impossible’ to have some sort of generalizable system that does this for all sorts of programs. I’ve heard people say it can’t be done, and code is the ideal format. I hold that in abeyance, I don’t really know, but intrigued to discover those who have a counter perspective to that and have attempted to build something. Also really curious if you know other similar people to Bret Victor I should check out!
Story Published at: January 23, 2023 at 03:49AM
Story Published at: January 23, 2023 at 03:49AM