Why

These are exciting times. More than 2³² people have digitally connected, and how we network is due an upgrade.

The internet of streams

The digital world you know is built on structuring streams of data. At its core the internet¹ attempts to provide a model where:

For any two connected devices running any application, there exists a connection to transmit data.

To do this it uses the following types of packets to transmit data between two devices.

Packets are transmitted and - using the IP address - they reach their destination; a device such as the one you’re using right now. On the device an application is running that waits for packets with a specific port to arrive.

In transit, a packet can get lost or corrupted. The result is that packets don’t arrive in the order they were sent. By adding a sequence number, the destination can verify all packets have arrived and reorder them to the order they were sent in.

I imagine the proto-internet was first discovered when people realized that packet SEQUENCE IDs are essential complexity. When transmitting data, something must define the order of packets. As a consequence, the physical route each packet takes is irrelevant.

The result is that conceptually each application on each device can talk to any other application on any other device.

This model is ideal for phone-calls or video streams. To build more interesting applications we create protocols to add further structure to the stream of data. There are thousands of different protocols, but what most of them have in common is a way to transmit questions and answers.

A couple of well known variants that have this property are:

System	Question	Answer
DNS	archive.org	207.241.224.2
HTTP	/forum/index.html	Hello world!
FTP	/Projects/linkspace/readme.md	In a supernet […]
SQL	SELECT * from MSG where ID=1;	A message in a db

At first we used these tools to create websites to provide information. But this has grown into interactive ‘platforms’. Services that store, forward, and administrate information such that multiple devices, and thus people, can communicate.

One reason for building linkspace is this:

We have reached the limit of using this model of streaming questions/answers paradigm for multi-party communication.

First, the stream abstraction can leak. Streams get disconnected, one side is overloaded or errors, etc.

Then, when more than two devices are involved the leaks compound. Synchronization is as slow as the slowest link, answers become stale or bad, and assumptions that an application relies on are no longer true.

Alternatively you built a supernet.

Once you start thinking in terms of supernets, it becomes clear that many problems with streaming questions/answers are accidental complexity. Using a supernet to build an application those problems become irrelevant or trivial² - and we can do things that are practically impossible if we keep talking streams.

Linkspace

Linkspace attempts to provide a model where: for any group running any application, there exists a space to address data³.

If the current internet is streams for key-value systems, so you can talk to server, then linkspace is a shared key-value space, so groups can talk about data.

A unit in linkspace is called a point. Each point has data, some auxiliary fields, and is uniquely identified by its hash.

To understand what each field does lets start with a simple example of a message forum.

The “image/BrokenMachine.jpg” is called a spacename and maps to (image data). So far this should look familiar as it is similar to files in directories. A spacename is similar to a pathname⁴ or event channel. In linkspace each entry is called a point, and i’ll refer to multiple entries as a set. The example shown has two sets merging. The result is a new set with 3 messages.

One of the most useful aspects of linkspace is a way to talk and think about digital communication in terms of a set of points instead of connections.

Practically any digital communication can be understood as merging sets of points.

Online platforms have dubbed different words for actions you can take: ‘creating posts’, ‘uploading image’, ‘upvote/like a post’, ‘stream a video’, etc. Fundamentally they can be understood as a frontend application providing an interface to merge sets of points. Either on your device or “in the ☁cloud☁️️” (i.e. on their device).

The internet in use today has a single host design. For instance, a web-browser or app contacts http://www.some_platform.com for the key /image/BrokenMachine.jpg to get an image.

This is relatively straight forward, but it has downsides.

A host can get disconnected, you can’t (re)share and (re)use your copy of the data, there is a misconception on what an address is⁵, and there is no standard on what happens when two people create two different image/BrokenMachine.jpg but with different pictures.

This might seem benign, but it has an enormous complexity cost.

With a single host design every server can define (and edit) what the ‘real’ copy of the data and its address is. Consequently, every device/server has slightly different semantics on how it deals with the issues listed - and most are missing basic features such as an unambiguous way to address things.

In linkspace there is no such thing as a ‘real’ copy on a single host.

Every spacename can refer to multiple points, and every point is hashed. There is a 32 byte number that uniquely addresses the exact content of that point⁶. (which I’ll show as [HASH_0] instead of typing out).

It doesn’t matter when or where sets are merged, a set only ever has a single copy of a point. It doesn’t matter how you received a message or how you send it; If you create a new message and refer to the hash, any device can knows exactly what you’re pointing to.

Anyone can uniquely address a specific point by its [HASH_0], or multiple entries through a spacename “/thread/Tabs or spaces/msg”.

This might seem more trouble than existing solutions like a filesystem or HTTP. In those, a request using a spacename gets you a single result. But this is not the case.

In linkspace points have a ‘create’ timestamp and can be signed by a public key (which i’ll refer to with [@:…keyname]).

The single result HTTP or a filesystem provides is an incomplete abstraction. It is unable to capture events over time, scaling is impractical, and a valid & consistent results is an assumption - that can break to disastrous effects.

In so far as that having a single result is simple; the same thing is achieved in linkspace by requesting only ‘the latest point with spacename X’ or ‘the latest point with spacename X signed by someone you trust’.

The effects of the second option is worth considering for a moment: When an application limits itself to points created (or linked) by a specific key, then that key has a administrative ‘include/exclude’ power, but it is not tied to a physical server hosting data.

Put in different terms - linkspace has the property that ‘hosting data’ and ‘data administration’ are decoupled.

A point has two preceding fields. A group that signal who is allowed to read or add a point, and a domain field to indicate which application uses it. Essentially any pair of (domain, group) has its own space.

For example the msg_board application and the [#:example] group.

Note that the group and public keys are 32 bytes. For brevity I used the LNS representation for [@:alice:sales:example] and [#:example]. LNS solves a similar problem as DNS, i.e. instead of using a number like 192.168.0.1 you can use the name www.example.com.

Finally, in our example so far I used a [HASH] directly in the data field. This would not work well for most use-cases. Instead, a point in linkspace has a list of links adjacent to the data.

Click here to see (part of) the example in graph form

Note that the ‘data=(image data)’, ‘data=[link:reply to]’, and the tags ‘reply to:’, ‘image links:’ are just examples. Each application defines its own data format and what tags mean.

For the full specification of creating and writing points see the guide

There are some nuances and various advanced topics. However, this should be enough to reason about the basics:

A single unit in linkspace is a point, addressable by hash and spacename, and links to other points. Domain applications provide a user interface to read and write points. Users generate their identity and form groups to exchange points.

The result is that conceptually an application only needs to process its view of a shared space.

To give linkspace a try Download the latest release, and check out the examples or the (python) tutorials.

Q&A

Who is linkspace for?

Linkspace was build to solve a problem for: Me, people who want a better internet, and everyone else.

Starting with that last group:

The tool

If you’re just interested in the tool; linkspace can shrink a ‘stack’ significantly.

Data formats change and grow as projects go on. By using linkspace to carry the data, many potentially-critical properties are there by design.

Instead of risking a costly and half backed upgrade later, you get them without any intrusive design changes required: uniquely addressable/cross referencing events, security against tampering, logging, cryptographic authenticated and auditable events, low effort backups, etc.

Linkspace can be scaleable decoupled data creation, routing, storing, and processing; with data authenticity and user authentication to control access, wire up business logic, or log ‘who’ did ‘what’ and ‘when’. Take the parts you want and leave the rest.

It even runs in the browser⁸, so you can reuse more code logic and tools.

Finally, there is a lot of potential for a ‘network effect’. e.g. a support-ticket references the exact IoT event, a sale can references the exact history of the object, an ecosystem of additional tools that talk (part of) the same protocol, and more.

The idea

I build linkspace so I could:

• define, prototype, build, and run real-serverless⁹ apps quickly.
• build an application without having to decide how to administrate for everyone that uses it - that responsibility sits with the (group of) users.

I believe it can be part of the solution for some of the worst problems with the current web.

Perhaps the greatest insanity of this time (and I would argue its defining feature) is a few for-profit advertisement / influencing services are the host-administrators of the digital ‘public squares’ for more than 2³² people alive.

It is questionable for people (and the groups they organize in) to be subjected to the interests of an unaccountable administrator. But that can be a matter of perspective, as you could argue people and groups could reorganize if the host-admin misbehaves.

The hidden cost is far more worrying in my opinion. It is the medium as the message: You are not in control, surrender your minds for profit and submit to apathy.

It is a staggering amount of wasted potential for people to never experience what it means to build and be a community outside the limitations and interest of a higher-authority-incorporated.

We are all lesser for it.

The message of linkspace should be: You are in control, and you should build the best places you can.

Is linkspace a blockchain?

No.

I have no tokens to sell - (But please consider support the project by registering an LNS name. Thank you!)

Blockchains and supernets share a common idea:

Using cryptographic hashes and public keys to provide a model for communication data that transcends the connection used to share it.

Blockchain use the cryptography to batch multiple events into ‘blocks’. As new blocks are made they are linked or ‘chained’ to the previous block. This creates a consensus of all events that happened.

A blockchain’s goal is to provide a model and tools to make consensus simple.

However, consensus on all events isn’t that useful as a foundational feature for the vast majority of applications.

Supernets don’t bother with a global truth. Their goal is to work with the links between packets. Consequently, it’s not difficult to define a ‘blockchain’ style consensus in a supernet.

Won’t an app have a lot of overhead compared to a basic Web server?

If all you want to do is stream one movie from a single host and forget it, then linkspace might be too much overhead. Few projects stay that simple. Most projects grow in scope: to identify users, save their comments, add them to groups, scale beyond a single server, etc.

Once a full stack is build, linkspace is very small w.r.t. its features.

As far as speed goes, linkspace point format is designed to be fast - Blake3, well aligned fields. The library is zero-copy where possible. A single thread on my machine can create new packets at a rate of around 800MB/s.

Can you ask people to deal with the added complexity?

Yes.

Linkspace lacks 6 decades of tooling that made the internet and web relatively easy for users, but that can change.

But is it worth it?

Yes.

The height of what we build is set by the weakest foundation. Technically supernets better model the reality of multi party communication - asynchronous and authenticated¹⁰ In the long run it could end up with less moving parts and with fewer configurations.

And finally, I believe it can do some good.

Wouldn’t it be bad if no one administrates what I and others see online?

Sure - to some extend.

There are different ways to do so. The straight forward approach is to have the application or user trust the public key of someone to scan and whitelist content. This emulates the current system of ‘admins’, while still having users give the option to replace them.

Alternatively can be more dynamic. You could have friends or friends-of-friends vouch for content.

I suspect the latter to become more important as AI drives the cost of bullshit to zero and while platforms have no incentive to identify the engagement and content from chatbots.

Won’t we end up with the same paradigm of highly centralized control?

Maybe, maybe not. If user today could walk away from a host-administrator without losing their history, identity, and links to others; then they would get a better deal then they do now.

Why not [alternative]?

There are two types of systems that can’t do what I want.

• Tokens / Synchronizing chains of trust: Is slow and not useful for the vast majority of users - (and easy to emulate in a supernet).
• Faster email: It’s not that fast and it is using server-defined authenticity - plus most of the list below.

Other supernet-like systems are limited in some way or simply choose a different design:

• Too specialized. E.g. Git has a lot of plumbing for diffing text.
• Large ‘packets’ - a hash might refer to gigabytes. This requires multiple levels to deal with fragmentation in multiple ways.
• Its hashes, without including spacename (i.e. paths, channel, etc) . Linkspace has both as a property of a ‘point’.
• Too slow. Packet routing/parsing should be doable in just a few instructions - ideally possible in an integrated circuit. It should be fast enough to stream video without using a secondary protocol to hand off to. That means no json or base64.
• No Groups. Setting who you share with and how is not supported, or its little more than ‘run multiple instances’.
• No domains. Everything becomes one app with premature-bureaucracy and an ever expanding set of compatibility requirements that grinds development/experiments to a halt.
• Its focused on signatures and consensus.
• Poor scripting support.
• Excessively interwoven components. e.g. Transmitting packets require a fully running ‘node’ with a fixed method of exchanging or saving data.

That does not mean I think alternatives are necessarily worse. Different systems have different strong points.

I think the two things that set linkspace apart from similar systems are:

• First class spacenames allow (part of) existing designs to be copied ‘as is’.
• All the things it doesn’t do. Linkspace is primarily its point format, and secondarily an ecosystem of tools to make it easy to build complex things with those points.

Linkspace is free and open source under the MPL-2.0. Give it a try next time you want to talk about data.