Programming for everyone! How computer systems learn to obey us on our word
Wouldn’t it be wonderful if we could simply describe programs using everyday language? During his appearance on the Campus Talks program (on ARD alpha), Sebastian Weigelt, Research Engineer at thingsTHINKING, talks about the future of programming and how it might be possible to let anyone – yes, really anyone – create simple programs. ProNat, which he developed, generates code from spoken process descriptions and shows what could be possible in the future if it is possible not only to process human language easily but also to really understand it semantically. Programming for everyone – this long-cherished dream could soon become reality… And maybe semantha can help us!
Broadcast: Campus Talks (on ARD alpha) – 22.11.2021
The content of the presentation in brief
Computer systems have become omnipresent companions in the everyday lives of many people. Whether it’s a laptop, smartphone or home automation system, we all use personal computer systems almost incessantly these days. However, for most of us, the focus is really on using existing functionality. The real added value of computer systems, however, is being able to program them. However, only a few people have a sound knowledge of programming. For many people, learning a programming language is an insurmountable hurdle. One solution to this problem is to offer non-professionals the opportunity to use everyday language for programming.
Thanks to the ever-widening use of digital assistants and similar voice-controlled systems, it is now completely normal for most people to talk to computer systems. While such systems now appear intelligent, they actually only respond to individual commands. You can’t program these assistants in the true sense of the word.
ProNat (short for Program Naturally), developed by Sebastian Weigelt in his dissertation, can convert spoken instructions formulated in everyday language into executable code. ProNat can thus be used to program computer systems of all kinds without having to learn a programming language. In addition, laypersons can teach the systems new functions by instruction. It is sufficient to describe the new functionality once in colloquial language.
Learning from instruction does not require extensive data sets with input and output examples. Instead, the user becomes the teacher and instructs the system by giving instructions on how to do something. ProNat generates code from the instruction and makes sure that the new function can be called afterwards, of course also by spoken instructions.
For example, the following scenario is conceivable: A user uses ProNat to explain work processes to a household robot, such as washing laundry or making coffee, for example as follows: “Robo, to make a cup of coffee, get the cup, place it under the dispenser, and press the red button”. ProNat creates a sequence script for the robot from this. If parts of the instructions cannot be interpreted, ProNat asks the user questions. The robot not only carries out the instructions, but also remembers the sequences. The next time it has to do the laundry or make coffee, it can call up the sequences it has learned. All the user has to do is call them up, for example: “Make me some coffee” or “Hey, I need cup of coffee badly.
Photo: ARD alpha / Bayerischer Rundfunk
