2006: Servers Started to Become Something Else
· 58 min read · Views --

2006: Servers Started to Become Something Else

Author: Alex Xiang

Essays

In the previous article about 2005, I placed the theme on “software beginning to live on the web”. By 2006, another deeper change was beginning to show its outline: servers were starting to become something else.

Before that, a server was first of all an asset. It sat in a machine room. It had a model, disks, memory, network cards, rack position, procurement process, and delivery cycle. To launch a system, you first needed machines. To expand capacity, you first bought machines. To build a test environment, you also had to find machines somehow.

After 2006, this understanding began to be rewritten slowly.

AWS launched S3 and EC2 that year. Looking back today, this is one of the most frequently mentioned starting points in cloud-computing history: object storage and elastic compute turned “storing files” and “requesting machines” into capabilities usable through APIs. For many engineers at the time, especially in China, this change did not immediately enter daily work. But it had already written out the future direction.

Servers were no longer only iron boxes in a machine room. They began to become resources that software could schedule.

I was still working in a Linux company then, doing relatively traditional systems, distributions, industry customers, deployment, and delivery. In our world, servers were very concrete: installation discs, partitions, drivers, kernels, service scripts, configuration files, customer sites, and machine-room environments. Open-source ideals were still there, of course, but more and more they landed on servers and on practical questions: can the system run stably, can it be delivered, can it serve customers?

That is why 2006 is a good year for contrast. On one side, domestic Linux companies were still trying to put open-source systems into servers and industry sites. On the other side, cloud computing was starting to abstract servers into higher-level services.

Servers Used to Be Heavy

Many young programmers today first encounter backend development through cloud hosts, containers, serverless, object storage, or managed databases. A few clicks in a console, or a few lines of Terraform, can bring up an environment.

2006 was not like that.

The “weight” of servers was very real. Procurement needed process. Machines took time to arrive. Operating systems had to be installed. Network cards and disks had to be recognized. RAID had to be configured. Machine-room networks had to be enabled. Remote management cards were not always pleasant. If the system failed, someone might have to go on site. People building distributions and system delivery were especially sensitive to these things.

Whether a software system could run was often not a code problem. It was an environment problem.

One customer’s machine model might differ, and drivers might differ. A slightly different kernel version could break a module. A deployment script that ran perfectly in the office could fail on site because of paths, permissions, character sets, or network policies. “Infrastructure as code” had not yet become an everyday concept, and much infrastructure knowledge lived inside engineers’ experience.

During my years at Linux companies, this sense of weight left a deep impression on me. You knew that a server was not an abstract noun. It had fan noise, temperature, disk lights, network cables, racks, and all kinds of uncertainty at customer sites.

So after cloud computing truly became popular, I could understand its value even more. It did not make servers disappear. It hid much of the pain of dealing with servers into deeper layers, allowing application developers to work at a higher abstraction.

S3 and EC2 Turned Infrastructure Into Interfaces

In March 2006, Amazon S3 was launched. In August, Amazon EC2 entered public beta. One was object storage. The other was elastic compute. They later became the foundation of AWS and one of the most important metaphors of cloud computing: infrastructure can be used through interfaces.

The meaning was not merely “renting machines became easier”.

In the past, when you wanted to store files, you first thought about disks, directories, file servers, and backup strategy. S3 abstracted this into objects, buckets, access control, APIs, availability, and capacity. You no longer directly cared where a disk was. You started to care whether the object was readable, whether permissions were correct, how cost was calculated, and whether latency was acceptable.

In the past, when you wanted to run programs, you first thought about buying servers, installing systems, and configuring environments. EC2 abstracted this into instances, images, sizes, start, stop, and scaling. Machines still existed, but to the user, they became a capability that could be requested and released.

Once this abstraction held, software architecture changed.

Systems could start small and scale after traffic grew. Test environments could be created temporarily and destroyed after use. Storage capacity no longer had to be planned too large up front. Businesses were no longer completely blocked by procurement cycles. Later ideas such as automated deployment, auto scaling, microservices, containers, DevOps, and serverless were all connected to this infrastructure abstraction.

Of course, Chinese companies in 2006 could not widely adopt AWS immediately. Network, cost, awareness, compliance, and ecosystem were all immature. But technology history often works this way: something appears far away first, then slowly seeps into your surroundings years later. By the time you really use it, you discover that the industry’s logic has already been rewritten once.

Hadoop Reminded People That Data Was Also an Infrastructure Problem

Google published the MapReduce paper in 2004, and Hadoop began entering the Apache ecosystem in 2006. That timing was important.

If S3 and EC2 changed how computing and storage were acquired, Hadoop pointed to another pressure: data scale was becoming too large to be solved only by traditional databases.

After internet services moved online, user behavior, logs, search, clicks, relationships, content, advertising, and recommendation all continuously generated data. Data was no longer only a few core tables inside business systems. It spread to system edges: access logs, error logs, user behavior, content indexes, message queues, offline reports, inverted indexes for search, and recommendation features.

When there is enough data, engineers quickly face several questions: where to put it, how to process it, how long processing takes, how to rerun after failure, how to validate results, and how to let more business teams use it.

Hadoop later went through both hype and decline, and it was replaced or supplemented by many newer systems. But at that time, it gradually brought “large-scale data processing” from the internal capability of a few large companies into the broader engineering world.

Only later, when I worked on a big-data platform at Weibo, did I enter this river more directly. Looking back, Hadoop in 2006 was like a stone that appeared upstream. The water would flow around it, wash it, and cover it, but it did change the riverbed.

The new internet technology stack starting to form in 2006

The Frontend Was Also Becoming Lighter, and jQuery Was the Glue of That Era

Another thing from 2006 looks plain today but was very important then: jQuery was released.

If you did not experience early browser-compatibility problems, it is hard to understand why jQuery made so many people comfortable. Writing JavaScript at that time meant obvious differences between browsers, tedious DOM operations, troublesome event handling, and awkward AJAX calls. The value of jQuery was that it wrapped much of this pain inside a very light API.

It was not a frontend framework in today’s sense, but it was a shock absorber for frontend developers of that era.

This echoes cloud computing in a subtle way. Cloud computing wrapped server complexity into interfaces. jQuery wrapped browser differences into interfaces. They worked at different layers, but the engineering philosophy was similar: put repetitive, fragile, error-prone things behind an abstraction so developers can express business intent faster.

Later, the frontend world would enter a much larger framework cycle: Backbone, Angular, React, Vue, TypeScript, build tools, components, and frontend engineering. But in 2006, jQuery had already made many people feel for the first time that browser-side development did not have to be so awkward.

When I wrote that JavaScript butterfly in 1997, my strongest feeling was browser chaos. By 2006, frontend development was still far from today’s complexity, but it was moving from “page effects” toward “application development”. That road was long, and jQuery was a key paving stone.

Twitter and News Feed: Real-Time Streams Started Changing Attention

Twitter appeared in 2006. Facebook also introduced News Feed that year and opened to a broader user base. Together, they pointed to a product form that later became extremely important: the information feed.

Early internet content consumption was more like “I go somewhere to look at things”. Portals, forums, blogs, news sites, and search engines were all actively visited by users. Feeds began changing that relationship: content continuously flowed in front of you, and users refreshed, stayed, interacted, and reposted inside the flow.

From a product perspective, this was a user-experience change. From an engineering perspective, it was a system-pressure change.

Feeds mean higher-frequency writes, more complex distribution, stronger real-time requirements, heavier ranking and recommendation, and finer user-behavior data. You cannot treat it as only a list page. Behind it are relationships, time, interaction, caches, push, anti-abuse, content quality, and growth mechanisms.

Twitter was still small then, and Facebook News Feed caused controversy at first. But later, the entire mobile internet would be reshaped by feeds. Weibo, Toutiao, short video, live streaming, e-commerce recommendation, and knowledge communities all developed around feeds to varying degrees.

This line has a deep connection with my later experience. After 2012, I joined Weibo and worked on the big-data platform and Weibo Toutiao. Only then did I really stand on the battlefield of feeds and recommendation systems. In 2006, it looked like changes in a few new products. A few years later, it became the main battlefield of the entire industry.

Java Open-Sourced, and Open-Source Ideals Entered a More Complex Business World

In 2006, Sun announced that Java would be open-sourced under the GPL. This was symbolically important.

Java was not a small project. It was already a major force in enterprise software, server-side development, education, and tooling ecosystems. Java becoming open source showed that open source was no longer only about operating systems, compilers, scripting languages, or hacker projects. It was entering the core of large commercial technology platforms.

But open source entering the business world did not make problems simple.

The Linux company I worked for was inside this tension. We believed open-source systems were valuable, and we believed domestic Linux had a chance. But in real delivery, customers cared about whether the system was stable, who took responsibility when problems happened, whether existing software could run, whether drivers were supported, and who would bear training and migration costs.

On servers, open-source ideals become very plain: can it install, can it boot, can it connect to the network, can it run the business, can it be maintained for three to five years?

This is not romantic, but it is real.

Later, I increasingly felt that the hardest part of open source is not “making code public”, but building long-term collaborative trust. Users need to believe the project will be maintained. Enterprises need to believe service is guaranteed. Developers need to believe their contribution is meaningful. Commercial companies need to find revenue models. If any one link breaks, open source easily remains a slogan.

Java open-sourcing, Hadoop growing, Linux commercializing, and cloud services rising in 2006 show, when viewed together, that open source was becoming part of the infrastructure world. It was no longer only a banner for idealists, nor only a tool for companies to save cost. It was becoming the foundation of modern software industry.

What Domestic Linux Companies Missed

Looking back at that group of domestic Linux companies, it is easy to say simply that they “missed cloud computing”. There is some truth in that, but it is also too easy.

The domestic market environment, customer structure, capital patience, talent flow, and position in international communities at the time did not support many companies directly turning toward cloud services. Distribution companies were good at operating systems, integration, adaptation, industry customers, and delivery. They were not necessarily naturally good at large-scale online services.

More importantly, cloud computing was not as simple as “selling Linux on servers”.

Cloud computing needed resource scheduling, billing systems, consoles, APIs, automated operations, highly available architecture, scaled operations, customer self-service, and long-term investment. It was a service capability, not an installation disc.

For many domestic Linux companies then, this was a completely different capability model.

What we were doing at the time was more about “getting Linux into customer sites”. That direction was not wrong, and it did have value. But the larger industry trend was to push operating systems and servers further downward into part of a cloud platform. What users ultimately bought was not a certain distribution, but the stability, elasticity, and efficiency of upper-layer services.

Many companies did not lack technical ideals. The layer where those ideals lived was gradually being rearranged by the industry chain.

The file-system-level work I was doing then also reflected this gap. SyncFS and SnapFS were close to customer sites: synchronization, snapshots, recovery, backup, and sharing were rigid needs in server systems. They were not glamorous, but they were critical. If file state is unreliable, no beautiful upper-layer interface matters.

But cloud computing was moving these capabilities further into platforms. Users did not necessarily care what the underlying file system was called, nor which module completed a snapshot. Users cared whether they could click once to create a snapshot, whether backups could run automatically, whether recovery could cross machines, and whether service could continue after failure. Infrastructure was moving from “you understand its internal details” toward “you call the capabilities it exposes”.

Around 2007, this layer change would appear around me in another form. The company was later acquired by an American company, and I began using C++ and GTK to build Linux-based cluster-management software. That C++ framework had been built by very strong overseas researchers and already carried many ideas that became common only later in modern C++, such as smart pointers, resource-lifetime management, object composition, and stronger abstraction boundaries. Later still, when the company could no longer continue, it once planned to sell itself to Huawei, but the U.S. government blocked the deal. That story belongs better in 2007, because it was no longer only a technical issue. It was where technology companies, open source, infrastructure, and international politics intersected.

Infrastructure Was Eaten by Software

The real meaning of 2006 may not lie in any single product. It lies in the fact that infrastructure began to be reorganized by software.

Servers were organized into elastic compute. Disks were organized into object storage. Logs and web pages were organized into big-data jobs. Browser differences were organized into JavaScript libraries. Social relationships were organized into feeds. Enterprise language platforms began entering open-source collaboration more deeply.

These things looked scattered at the time: AWS was a cloud service, Hadoop was data processing, jQuery was a frontend library, Twitter was a social product, and Java open-sourcing was a platform event. But over a longer time scale, they were all doing the same thing: turning things that were heavy, scattered, and hard to operate into capabilities that software could schedule, reuse, and combine.

This would change the work style of all programmers afterward.

Backend engineers no longer faced only one machine. They faced a set of scalable resources. Frontend engineers no longer only wrote pages. They built application experiences. Data engineers no longer only wrote reports. They processed continuously generated data at scale. Operations engineers no longer only installed systems and watched machines. They wrote automation and platform capabilities.

In 2006, I had not yet fully entered these new battlefields. I was still in a Linux company, writing code, changing scripts, dealing with systems, and working with open-source components. But looking back today, that year had already buried the later route very deeply.

Servers started to become something else. A few years later, developers would become something else too.

IT Events of 2006

  • Amazon S3 launched. In March 2006, Amazon S3 was launched. Object storage turned “reliably storing large amounts of data” into an on-demand cloud service, becoming a foundation for later cloud computing, data lakes, and internet applications.
  • Amazon EC2 entered public beta. In August 2006, EC2 further abstracted servers into computable resources that could be requested, released, and billed. Infrastructure began moving from machine-room assets to APIs.
  • Hadoop entered the Apache ecosystem. Hadoop brought ideas from Google’s large-scale data-processing papers into the open-source engineering world. Big data was no longer only an internal capability of a few companies. It started to become infrastructure available to more teams.
  • Twitter launched. Short messages, follow relationships, and real-time streams combined to change the rhythm of information distribution. It later influenced Weibo, feeds, hotspot propagation, and public-discussion product forms.
  • Facebook News Feed appeared. News Feed pushed social networking from “a collection of personal pages” toward a continuously refreshed information stream. It later became a central form that social products, content distribution, and recommendation systems could not avoid.
  • jQuery was released. jQuery used lightweight abstraction to smooth browser compatibility and greatly reduced frontend-development complexity. It represented a key step as web frontend development moved from scripting tricks to tool ecosystems.
  • Google acquired YouTube. Google acquired YouTube for $1.65 billion, further confirming the importance of online video. The relationship between content platforms, advertising systems, and infrastructure became clearer.
  • Java became open source. Sun announced that Java would be open-sourced under the GPL. Open source was no longer only a community movement. It entered the core of large commercial technology platforms and influenced OpenJDK and the enterprise Java ecosystem.
  • Cloud computing rewrote infrastructure. Around 2006, storage, compute, data processing, and web applications began to be reorganized. Servers were no longer fixed machines. They gradually became platform capabilities and programmable resources.

References

More in this column

View all posts in "Thirty Years in IT and Me"