There are a number of theories for why we've had a divergence between wages and productivity in the past forty years. Some point to technology, others to the fact that there is now a massive "reserve army" of labor as China, India, and other large populations previously shut out of our international economy have joined it in force. There's probably some combination of both as jobs that can't be shipped overseas become the target for automation investment while those that can be sent abroad are because that's the easiest thing to do. Either way, the majority of classes of labor have lost leverage in negotiating a greater return while capital has gained greatly. And just to be clear, I consider automated workers, rather physical or virtual, to be the perfect example of “capital” — non-labor support of production.
As an engineer, I of course have an interest in the technology part of the equation. I may actually be contributing to this trend or will in the not-so-distant future. I have been thinking about this for a while and wanted to share those thoughts. I know I am not the first to speculate on the “machines are coming for our jobs” story by far. What I do hope to contribute is some background on the various technologies in play to help ground the discussion. I tend to find the other articles on the web too reliant on anecdote as in “hey look, here’s a thing in a lab that looks like something we want to talk about” rather than giving a perspective more based in cause-and-effect.
I think the world has the potential for more radical changes than we may realize. There are four parts to this tour which can be broken out as follows:
- Where do the machines' powers come from?
- What might we expect to see machines able to do?
- Worse than cyberpunk: Software sorcerers and skill-less serfs - the dark path
- Paradise found: Collective Cornucopias - the bright path
Part 1 and 2 of course sets the range of travel for parts 3 and 4. I'm sure those more socially inclined than me might be able to extrapolate the cultural bits more accurately than me. But I'm still going to have fun doing it.
Part 1. Where do the machine's powers come from?
The story of machine capabilities is one of exponential growth. Unlike the short human lifespan, the durability of our common knowledge ranges over centuries. We learn new physics. New processes are invented. Each new tool makes it that much faster to create the next one. Achievements that put names in history books for one generation are mere recipes to the next.
The exponential power of growth was first applied to brute force efforts like moving
water or grinding
wheat. It soon became comparatively more delicate through supporting chemical production
in mass quantities and
textiles, and then
bridges and railroads as the materials became available. It became personalized as labor-saving devices like washing machines as it reached the home in
eras of prosperity.
But all along the way, the nature of machine power was almost solely physical. Every machine needed human pilots to keep humming. The massive bureaucracies, both
public and
private, needed to keep the supply chains going were totally human endeavors. The machines seemed to require as many people to feed them, organize them, and keep them going as they displaced. Each increase in brute output required people to guide flows of material and information to the right places to keep forward momentum going.
Then we got to software. Technology is no longer purely about applying force or manipulating matter. It can also manipulate rules and plans along the way. The exponential progress is quickly tearing its way up the list of tasks ordered by complexity as it once did scales of available matter and energy.
There is a point at which all of that mass-and-energy manipulating technology and all of that task-and-process manipulating technology will come together. Once we get to that point, we’ll be able to see our technology able to do a huge array of things without human intervention.
"Exponential" is invoked as a scary, hard-to-see-until-it's-too-late term. On the other hand, we have heard this story from General Ludd and his many, many intellectual heirs before without disaster totally striking. I promise here to stay away from the wide-ranging social impacts until the next couple of parts. It will be about just the facts as far as I can see them from here on for this part.
A short history of software and its exponential growth
Let's talk about software. In its humble beginnings, it was a totally rote exercise. Flip this switch and a cascade of other switches will tumble into the off or on position just like so many dominoes. That cascade is a totally deterministic process, where all possible combinations of switches can be written out. These are called “
truth tables” amongst computer scientists. Once upon a time, computers were literally
controlled through these switch flips. Tapes or cards served as improvement by letting operators record and replay their instructions. Even better, libraries of these choreographies could be built up and shared.
A lot of progress in the efficiency of specifying software comes from “abstraction.” The native form of computer thinking is the flipping of switches and logical operations. Not only is writing in this format slow, it is very difficult to understand why a given program was written. This makes sharing and leveraging others’ work very hard. Abstraction in programming is about moving away from what the computer is doing and more toward the why of the program. Ideally we would program in recipes like “take the car downtown to pick up my dry cleaning” which embeds the purpose of the action.
The first level up in abstraction were
mnemonics that annotated the binary addresses that stored these operations in the main processor with names to help programmers rennet what they meant. Of course, this was not a very large step in abstraction — it just made you able to group the 1’s and 0’s on the chip somewhat logically. Developing in this framework continued to be a very low-level enterprise that requires a lot of knowledge or theory to perform effectively. A great deal of highly-skilled human effort and patience was needed for these programs, keeping them small. Also, the location of operations and their nature were of course proprietary to each machine and required relearning to transfer programs over as hardware evolved.
The 60's and 70's gave us the first big waves of innovation as the computer programming community hunted to improve the way they specified programs. They wanted higher levels of abstraction to make their program intent more clear than a series of address-based operations and switch-flips. Also, the simple act of writing programs in logically- and mathematically-oriented computing languages made them much more readable than a series of notches in paper. The other big thing this community achieved was portability. Standard languages like
C allowed the programmer to move his or her work from computer to computer and thus make programs that much more reusable.
The many intervening decades took these developments and refined them greatly. Programmer productivity and the ability to naturally and correctly specify code were big drivers. Larger programs required teams of developers and so languages evolved to help share the burden. This required more readability, the ability to decouple the work of one developer from another, and many checking and debugging tools as well as tools like source version control to carefully track contributions to the code base. Productivity has since skyrocketed and there are many pieces of software with more than a million directives within them. And of course, larger programs require more computing resources, which Moore's law dutifully supplies.
In modern times, there are a lot of different languages and approaches. But, a vast majority of software today is not fundamentally changed from predecessors over the past decades. This software is fundamentally a flow-chart or a simple checklist procedure. If you see this, do that. Memories of previous actions are built up during an individual execution which allows for a fair amount of complexity in the behavior of the program. But fundamentally, this is rote instruction for the most mindless drone: I will tell you every situation you will encounter, and will tell you exactly what to do in each case. This is called the
imperative paradigm of programming. A great number of software "bugs" are in fact perfectly described behaviors that just don't match human expectations. I tend not to get too upset with these things - I can't predict what I will want at any given moment either. Of course, your typical user is not so philosophical (cough, cough: the DK5 roll-out).
So most of our modern computers are blind rule-followers. What about the few that I didn't mention? Here's where things get more interesting.
Moving toward something we might call “intelligence”
In dealing with other people, you expect to be able to give a somewhat vague direction and have them draw upon their knowledge and skills to bring what you want. Ideally, we'd like computers to do the same. They are slowly getting better at that.
Classification
Problem number one is classification. To effectively deal with a situation, you need to know what it is. People do this all the time. For example - are mobile web sites different than "traditional" ones? Depends on who you ask. But if there is a difference, it suggests there may need to be differences in format or interaction approach.
Computer science has made great strides in this one recently. Admittedly, many of the services described here are imperfect but there is a lot of effort to improve them. There are codes that connect the sounds of speech with their written symbolic
equivalents. Google has a slew of image recognition algorithms it is currently
building. Microsoft can tell that my nasally uttered words are the same as your drawled
sayings. It goes one level higher than that using semantic technology, aka the Semantic Web.
To see semantic technology in action, Google Natalie Portman. Google now knows she's an actress. It also knows that actresses tend to be in movies and TV shows and plays. It knows Natalie Portman is a person, and might have some interesting family members or romantic partners. It also knows that when you look for Natalie, you also often ask about Garden State. And thus it will present you with co-stars.
What you have to know is that the way Google returns these results to you is fundamentally different than how they did it back in the early 2000’s. Back then, frequency of a word in text was a big part of understanding that two sites might be talking about the same thing. The new Google searches for well-known people like Natalie are no longer presented to you by matching search text to page text. Semantic technology captures abstract concepts like Actress and their conceptual connections to things like Acts of Performance Art. By using these abstract relationships, it doesn’t need to wait for two people to write different articles on Black Swan and Garden State, mentioning the actress by name. It already knows.
One of the more interesting Semantic Web sites on the web is
Wikidata. It uses collaborative editing just like Wikipedia. In fact the two are cross linked — Wikipedia pages can be used to annotate Wikidata entries and the summary boxes on Wikipedia articles are now being pulled from Wikidata. My favorite feature is how properties of things are defined. Rather than being absolute facts, things like the population of a city are qualified with sources so that we know the level of confidence that should be had in a given value.
Understanding intent
Problem number two is declaration of intent. Or in other words, you try to tell the computer what you want it to do clearly enough that it knows when it has succeeded. This one has been worked on a lot longer than the first. Rule-based systems and formal logic have been a big part of computer science
for a while. There have been more than one wave of enthusiasm and disappointment that these systems would bring us into an era of artificial intelligence. Even though rule systems did not bring us HAL9000, they are still very useful things.
The rule-based style is quite different from the imperative programming approach. The imperative, as said earlier, is a handholding recipe for every step of the way. Rule-based approaches are more of a "here's what I want and I don't care how you get it." It's a lot closer to how people ask each other to do a job, although we use natural languages to do that and they provide a lot more ambiguity. Imagine a robot car delivery service (coming sooner than you may think). To use it, someone needs to describe "drop off point" to the car. One rule for this is "Your GPS location must be within 50 feet of the customer's destination." That sets an unambiguous goal. Of course there is the possibility that this is physically impossible. Maybe it should be "closest point to destination that is also a point on a legal driving path where you are allowed to stop" but that is part of the art of defining these things. A big part is understanding what rules can be valid, and which can be checked for conformance by the machine. Machines are also really good at following directions, so a lot of the cleverness is in understanding and checking how a literal interpretation can go horribly, horribly wrong.
These rules are formally expressed in mathematics. To be effective, you have to know a pretty good deal of abstract math like set theory. This actually points to a big issue with using these systems - your average programmer has been trained in and is much more comfortable with the recipe-based approach. This isn't a permanent obstacle of course but does add some friction to the system. Take my “get as close as you can while staying on the road,” for example, in some rule-based psuedocode:
DrivablePath {
memberPoints belongs to Set of Spherical_Surface_Point
function onPath(somePoint is Spherical_Surface_Point,
otherPoint is Spherical_Surface_Point) returns Boolean {
if somePoint.lat = otherPoint.lat And
somePoint.long = otherPoint.long then true else false
}
}
Spherical_Surface_Point {
lat is Real
long is Real
function distance(otherPoint is Spherical_Surface_Point) returns Real {
* calculate distance on a sphere’s surface — code left as exercise to the reader
}
}
DriverRoute {
paths belongs to Set of DriveablePath
startLocation is Spherical_Surface_Point
enteredEndLocation is Spherical_Surface_Point
drivingEndLocation is Spherical_Surface_Point
rule setDestination {
drivingEndLocation is in paths.memberPoints And
* here’s the core rule that checks minimum distance
for all point in paths {
enteredEndLocation.distance(drivingEndLocation) < point.distance(drivingEndLocation)
}
}
}
searching for the answer to the problem
Now we know how to label things and how to set goals. What next? Well, the third big problem under assault right now is called "search" in the contexts I've seen it. This is different from search as you've likely come to know it from engines like Google. In this case "search" means "search for a solution." Take chess as the example for this one. I want to find the next best move. Chess has a finite number of moves, so it is easy to calculate the future possibilities or "space" as it is often called. I have a finite set of moves, then my opponent, then me and on and on. The longest game ever played at championship level was 124 moves, so there's also a finite set in that direction in practice.
The dumbest kind of search is try all the possibilities, like finding someone's phone number by pressing 111-1111, 111-1112 etc. There is also random search, where you simply throw out a possibility at random until you pay off (this one can be strangely effective at times). Of course, no one gets to write a paper on random guessing, so the experts tend to try more creative things. One of these for chess is called "
minimax" which is short for "minimize the maximum" or play conservatively. The approach is to minimize the worst-case scenario from each move. Calculating the "worst case" is of course the hard part and you have to forecast multiple moves ahead. Typically the worst case is letting your queen be maneuvered into a corner for capture within some number of moves.
Some of the approaches like minimax for chess can be quantified and characterized mathematically. This is the role of complexity theory, where algorithms are ranked by how many steps are required to compute a solution.
NP problems, or "non-polynomial" problems tend to be very expensive because their solution time grows according to a power law with the size of the problem. 2^n for example becomes a very large number very quickly even for small n. The goal is typically to find a polynomial-time or even a linear algorithm where time grows at a more manageable rate of n^3 or even n. Of course, even inefficient algorithms can get bailed out by Moore's law providing ever more computing power to attack the problem with.
Computer scientists tend to focus on theoretical problems and absolute worst case for their algorithms. Application-oriented engineers like me will often go for progress with "good enough" solutions. These are called heuristic approaches to mark their less-than-perfectly-formal basis. Examples of these are genetic algorithms, where partial solutions are mixed and matched like genes in the hopes of finding the complete answer. GA's are quite capable of solving real problems in often stunning ways. They have been demonstrated to be able to
"invent" circuits for example.
Modeling — Key to predictions
The final ingredient in our mad scientist concoction is modeling. That is my specialty within aerospace engineering: physical modeling. We've been doing this for a very long time. Young children do it - object permanence can be thought of as using a kind of model. Namely that objects don't just disappear, so if you can't see something you saw just a moment before it must just be hidden. Further, there is an understanding of Newtonian laws of motion. So if the ball rolls into view and then out again, one can expect to reappear at a given time.
Very young children grasp this and can be seen to be perplexed when adult experimenters cheat and steal the ball when it is hidden from view. Philosophers also spent a lot of time doing it in amazingly sophisticated ways. In fact they were so good at it that when they would get things wrong it would be
very hard to set them right again.
Our scientific revolution was all about a rigorous process to distinguish models. Once you had a model, like the Ptolemaic for the solar system, you could make predictions. A competing model could also make predictions. Those could then be checked against observations. And it is precisely when astronomical observations became sufficiently refined, the two models competed head to head and one was found
very lacking. The simplicity and accuracy of the Copernican model eventually won the day.
Models greatly accelerate the guess-and-check process of designing things. You can take a given wing design for example and subject it to thought experiments using your highly-trained mental models. For example, I can give you a rough idea of whether a given wing form will go supersonic nicely or not. For precision, it goes into the computer. The eventual "experiment" is less a trial and more just a validation that your model doesn't have a fatal flaw.
Developing models is an expensive process, and they tend to be highly prized by their owners. Experts will sometimes hoard their models and be bitterly defensive about being required to employ someone else's unless they can be inspected. Inappropriate use of models can be devastating — for example see the
Columbia accident report. The key is that models are fed entirely by data and the set of data must be rich enough that they encircle the various circumstances in which they are to be used. In technical fields, these data are expensive as they must be collected by highly trained experts using well-calibrated and controlled facilities. If I recall correctly, my university wind tunnel was a few
hundred dollars per hour to run, and a given characterization of an aircraft would take a week or two. For a more generalized model, many configurations would have to be run in a wide variety of circumstances to concoct the general approach. It's a long and unsexy process. But the product is very much worth it.
Bringing software progress back into the “real” world
All of the above factors talk about how process and intellectual machine power are surging forward. The most potent economic power in these trends is in the marriage of the physical and the virtual. That is the domain of industrial production in the large such as automated steel mills or in the small such as robots for piecework. Just as current software needs a lot of human handholding to define processes, industrial robots need
exquisitely controlled conditions. The thing to be worked on must be precisely pre-placed, whether it is a hunk of metal or a car. The thing must also match precisely predefined shapes and weights.
The area of research working to overcome these obstacles are in the area of dexterity. Sensor systems must become more precise individually and in concert. An internal representation of reality is needed to connect sensor inputs and actions taken to provide an understanding of how to get an object from state A to state B. These problems are being attacked on many fronts. Machine vision received a huge boost in sophistication and interest when camera-based approaches
won the DARPA grand challenge for autonomous ground vehicles. Just like with humans, machine vision works to take multiple two-dimensional views of the world and transform them into estimates of three-dimensional reality.
A big part of vision work is called “
scene reconstruction,” which is much like it sounds. In software, images are processed to develop a scene to match the real world. This scene aims to place shapes in space relative to each other. The first question is whether you can go through a given part of the scene or whether it is solid. The next is what can be moved or rearranged within the capabilities of the robot. Finally one must ask if parts of the scene are dangerous (fire, cliffs, mud pits) to the robot.
If the robot is to manipulate something, that is a
different bit of dexterity to deploy. Can something be handled without crushing it? How does one work with delicate operations like threading a screw? For us, there is a sophisticated sense of touch. Many are working to replicate this is laboratories with different degrees of success.
One more thing to consider is that the natural and even the urban worlds are full of uneven terrain. Wheels, our favorite machine locomotion, don't work everywhere. Our urban world is built for bipeds with two arms because that’s what we are. For machines to live and work with us they'll have to be
able to do the same (at least some of the time; quadrupeds with arms and the ability to reposition themselves might do fine). Standing and walking are activities that require amazing balance and a clever dance of shifting weight to achieve. Strides are being made there as well.
Summary: The exponential growth of the future is about making our machines easier to instruct
The real wrap-up I have to give here in the trends of software (and robotics) acceleration is in making our machines understand us and the physical world better. Rather than relying on people to input every step of the process, like you would instruct a complete imbecile, the new tools work in higher-level ideas and make the computer do some of the actual thinking. In a lot of cases, it is not worth the trouble to instruct computers on exactly how to do every little thing. That’s why a lot of jobs have so far resisted automation. As I’ve hopefully established, that trend could shift dramatically, and soon.
More on what this looks like when it is deployed in the next round.