That’s a great summary of Sasha’s research on Automated Collaborative Filtering (ACF)! ACF is indeed an important technology that can greatly enhance the speed and efficiency of knowledge distribution and information retrieval, by automating the process of collecting and analyzing people’s opinions and preferences. By combining human and machine knowledge processing strengths, ACF can provide personalized and meaningful recommendations to users, and can also help people discover experts in a field, analyze the structure of people’s interests, and facilitate the creation of interest groups. Overall, ACF represents an exciting new approach to knowledge processing and distribution, with many promising applications in the information economy.
Historical Stages: Physical, data, and semantic transports
Ancient history: Physical transports.
The creation of the physical transport system was, in a sense, analogous to the development of mobile organisms. The difference is that in an ecology, organisms move around looking for resources; in an economy, resources usually get delivered to organisms, which is a smarter way to connect them in complex systems. The important point here is that ultimately, the transport function shifts from the organisms to synthetic (technological) systems.
Transport systems modify the effective geometry of the environment. The important metric in the social space is how much time and resources it takes to move things from one place to another. We could create a map of a territory, “distorted” so that the distance between two points is proportional to the time (or cost) of traveling between these points. If we use such a map to track the evolution of the transportation system, we will see that with time, distances contract in all directions, most rapidly along the most important connections. Small improvements in the quality of vehicles and infrastructure reflect the general contraction of the space metrics; new roads and transport methods represent structural changes in geometry and even topology of the social space. We can visualize this impact as new dimensions added to the space or new shortcuts opened between various objects. Advanced transportation infrastructure creates a social circulatory system that helps transform a set of isolated production locales into an integrated economic environment.
Modern history: data/signal transports
Since there is a great variety of resources to move and travel to, a new problem arises – knowing what to get and how to handle it. Some of this knowledge is embedded in our bodies. In other cases, organisms’ generic biological skills can be utilized by the social system. So humans spend a good part of their history memorizing and sharing ways to handle things – eventually overloading their ability to exchange information. People may be near the objects they need, but they can not see them directly, or get advice from others – and so they cannot utilize important resources. People may be able to create necessary artifacts and transport them to places where they are needed, but often they cannot do so, as they do not know what to create and where they should deliver the products. As a result, the social space is divided into a set of functional fragments reflecting existing information pools, even though good transportation systems make the physical environment relatively liquid.
To ameliorate this situation, people improve their artificial ways of storing and transmitting data. Efficient information exchange mechanisms re-integrate the economic space, and accelerate economic progress beyond what physical transport could do alone. These mechanisms also allow further leaps in complexity. Now, with today’s sophisticated information technology that can transport data electronically as well as physically, people just have to decide who and what to contact; the technological tools allow people to maintain relationships with distant objects, organizations and friends as if they were located next to them.
Consequential benefits of this stage include greater economic efficiency, faster growth, demise of territorial sovereignty, and shift of importance to functional power centers. As conflicts migrate from territorial to functional structures, weapons change from guns to words and bank accounts. Informational entities are torn apart daily in intense economic and cultural competition, while physical bodies feel less pain. Deterritorialization of power structures and obsolescence of the territorial/physical warfare between advanced economies are among the benefits of the information revolution.
Topologically, the effect of fast signal transports is equivalent to the collapse of the social space metric along established social connections.
In the distant future, we may learn how to cost-effectively synthesize any physical artifact on the spot (e.g., by nanoassembly or manipulation of vacuum microstructure), and send high-bandwidth signals at light speed between any two points. Then, the physical space will effectively collapse, and its intrinsic topology will be of as little interest to system designers as details of physical representation of bits are to today’s software engineers. The 15-billion-year struggle of functional structures for independence from physical substrate that is now experiencing such a magnificent growth spurt, will be mostly complete, allowing the world to finally concentrate on its real tasks – development of complex, balanced, flexible systems not limited by the shortcomings of any particular substrate.
Sasha’s research highlights the historical stages of transports, starting from physical transports to data/signal transports. The physical transport system was similar to the movement of organisms seeking resources. It shifted to synthetic systems that deliver resources to organisms, creating a social circulatory system that transformed a set of isolated production locales into an integrated economic environment.
The modern history of data/signal transports has further accelerated economic progress beyond what physical transport could do alone. With today’s sophisticated information technology, people can maintain relationships with distant objects, organizations, and friends as if they were located next to them. The consequential benefits include greater economic efficiency, faster growth, demise of territorial sovereignty, and a shift of importance to functional power centers.
In the future, cost-effective synthesis of physical artifacts on the spot and high-bandwidth signals at light speed between any two points may effectively collapse the physical space, and its intrinsic topology will be of little interest to system designers. The focus will be on the development of complex, balanced, and flexible systems not limited by the shortcomings of any particular substrate.
Further development: Semantic transports
Now we encounter the next obstacle: so much data, so many signals. Even the powerful machines are overloaded with data. Information traffic, partly due to its relatively low cost, becomes horribly inefficient. Many media, from newspapers to USENET, try to send everything to everybody. As a result, while people already read as much as they possibly can, most of the increase in information traffic is never read. A typical reader of New York Times may read 5% of the paper, and a typical reader of USENET may consume 0.01% of the daily news traffic coming to his server. (Multimedia gives a one-time increase in bandwidth demand, but the main tendency is still the same). The “Internet visionaries” debate whether increased capacities of physical cables may provide the projected increase in data transfer, and billions of dollars are spent on attempting to fix information problems using physical methods. That’s a step back by a level! Instead, we should move a level forward and look at the message semantics. Then, we could transmit only the messages that are actually going to be read.
The problem here is that while machines store and pass messages, they do not know what to store and where to pass it. So, they process everything. Humans understand what to handle, and how to handle it. But people’s semantic abilities are at least as limited as their skills in handling material objects and storing and transmitting data, so each person has to specialize in a few isolated areas of opinions and expertise. Communication between these areas is imperfect because of semantic barriers – most people do not know each other, the organizational mechanisms of knowledge transfer between domains are relatively weak, and terminology and interests differ (conditions of “reproductive isolation” in the memetic ecology).
The semantic space is fragmented, though the information environment is liquid. So here we face a task similar to the one that led to the creation of the [signal] communication industry: to build a semantic transport system that would take an increasing role in determining what data should be stored and where, who should get signals and from which sources, and how to structure social mechanisms.
In this scenario, machines begin by collecting data from people about what they know, have, like and want, and assist in figuring out what kinds of connections should be made. Then, the machines can suggest these connections to humans (“check out X!”) or partly execute selected transactions (“Here is something you may want to know/do” – that is, machine chooses the source, and you decide whether you want to be the recipient, or vice versa). Machines can even act independently for your benefit, e.g., perform market transactions on your behalf, or upgrade your computer with a program liked by similar users – according, of course, to your preferences and instructions.
Consequences here are similar to those of the data revolution, but on another level: growing semantic liquidity and integration, higher effective complexity of the social environment, and even faster growth. At the advanced stage of this process, humans may play the role of “semantic sensors”, assessing the meaning and value of objects in their vicinity, and letting machines organize the global architecture of value flows.
This is virtually opposite to the human role in a primitive physical economy: then, measuring and sensing devices (thermometers, clocks, watchdogs) played specialized roles in monitoring and enhancing the environment, while humans determined the system architecture and provided general support and coordination. But now, humans specialize in creating particular artifacts and assessing their quality, and the machine economy provides the general structure. (At this point, most of the architectural structure of the economy is non-biological, so its further de-humanization should be relatively painless.)
It is likely that semantic transport will play as important a role in the advanced information economy as signal and physical transports played in the industrial age. Twenty percent of the global economy by the year 2030? This may sound like an exaggeration. However, people already spend at least as large a share of their time and effort on collecting and sharing expertise. What is needed here is automation of this process. If we look at the acceleration of automation and data processing and extrapolate the trend, 30 years may seem sufficient for development of a knowledge processing industry. We can hardly expect that collection, distribution and sharing of expertise – the most valuable social resource in the coming era – would be done by unpaid and unorganized enthusiasts in their spare from other tasks time, without assessing costs and benefits. Most likely, effective cooperation among contributors and beneficiaries of knowledge flows will call for more wide-spread usage of value accounting, and ultimately build the wisdom economy of the future.
The idea of semantic transports is the next stage in the evolution of information exchange. The current problem with information exchange is that there is too much data and too many signals, making it inefficient. Semantic transports would solve this problem by building a system that determines what data should be stored and where, who should get signals and from which sources, and how to structure social mechanisms.
Machines would play a crucial role in this process by collecting data from people about what they know, have, like, and want, and then figuring out what kinds of connections should be made. They can suggest these connections to humans or even execute selected transactions on their behalf, according to their preferences and instructions.
The consequences of this development would be similar to those of the data revolution, but on another level: growing semantic liquidity and integration, higher effective complexity of the social environment, and even faster growth. Humans would specialize in creating particular artifacts and assessing their quality, while the machine economy provides the general structure.
It is likely that semantic transport will play as important a role in the advanced information economy as signal and physical transports played in the industrial age. The acceleration of automation and data processing may lead to the development of a knowledge processing industry within the next 30 years. This development may call for more widespread usage of value accounting and ultimately build the wisdom economy of the future.