Our ultimate goal is to make sandbox worlds populated not only by thousands of players, but also by an organic simulation formed out of deep, interlocking systems with a life that can exist alongside players or without them. This simulation also needs to support user-generated content that enables creativity on both the small and large scale, and it needs to adapt to changing technology on as long a timeline as we can reasonably predict.

At the beginning, our essential challenge is that we are making software with no single purpose. While the vast majority of software is made to accomplish a task, whether simple or complex, we are instead attempting to make software that can be used for as wide a range of tasks as possible.

The goal is to make a system that is as close to essential building blocks as possible. We are not making a bowl, or even really making the clay that forms a bowl. We are making earth and water, capable of becoming anything else.

That means that the goal in developing our engine is to begin by making as few assumptions as we can about what the engine will be doing and how it will be doing it. At the outset of this project, our objectives fall into three main categories, all of which are inter-connected:

• Scalability: The engine needs to be able to support a complex simulation that involves a large number of objects and players spread out over a massive virtual space. It also needs to be able to continually grow over time, both in terms of complexity and sheer size. The system should allow data processing both inside and outside the engine, at runtime or not.

• Flexibility: The engine should support as wide a range of systems, interactions and tasks as possible. This should include tasks that we have not predicted and cannot specifically plan for at the beginning of development. On a basic level, we want to make a system capable of transforming data-A into data-B via processing, with as few assumptions as possible about the nature of either the data or the processing.

• Efficiency: This is both related to scalability and its own goal. The engine should run as efficiently as possible, allowing for many threads to be executed at the same time. This means that we are designing our engine to scale up to the maximum number of processing cores available to it in either the CPU or GPU. This should allow the simulation to run more efficiently both on modern hardware and hardware that has yet to be developed, assuming that hardware development continues its current trend of increasing parallel processing capabilities. Efficiency means we can do more things at once with less power, which means we can do more things overall.

The core concept behind how we are approaching these goals is data-oriented design, a development philosophy that attacks all of our goals at once. We are implementing that philosophy by building an engine around the idea of an Entity Component System, a concept that should allow for the grand scale flexibility we need for the simulation in Project Artemis. While we’ll go into more detail about exactly how our engine intends to use ECS, it’s helpful to have a basic understanding of what it is to understand why it works to start with.

Entity Component System Overview

The Entity Component System is what defines all of the individual things in our world. On a fundamental level, it is a way of treating everything in the world as the sum of its parts.

It is easier to understand ECS by what it is not, which is object-oriented design, a more traditional approach that you can see in lots of games. Object-oriented design works in categories, with every object inheriting all the traits of the category to which it belongs. So if we put a chair object into the world, it inherits all the properties that we’ve assigned to the chair category. This has its uses and makes it easier to add objects into the world, but it runs into problems when it comes to flexibility in particular, because that chair can only ever be a chair.

An Entity Component System, on the other hand, treats every object in the world as a unique entity defined by properties that it holds. Our chair becomes an entity with components such as a size and a shape, a position, some textures and the ability to sit on it. Depending on the complexity of our simulation it might have more, such as a weight, a material, a strength, a hardness, and so on: the list could go on. The entity exists as the sum of these properties.

If we want our simulation to find all the chairs in the world, we tell it to search for all entities that share certain properties that we have decided define a chair.

Object-oriented design defines objects from the top down, and an Entity Component System defines objects from the bottom up.

We can break down the concept of an Entity Component System into its three major parts:

• First, we have Entities, which are unique identifiers for items in the world. These entities are not defined on their own, they are just boxes that contain properties.
• We can attach Components to those entities, which are the data that define its properties. Any entity can have any number of components.
• Finally, we have Systems, which govern the behavior and interaction of those components.

For example, a physics system can look at all the given entities involved in an interaction as defined by their locations, pull the relevant components (things like size, weight, position and shape), and then run the equations necessary to determine how those components will interact. If the physics system decides that an object will move as a result of a force applied to it, it can change the components related to positioning.

Object-oriented design is a human way of looking at the world. When we look at our chair, we do not, at least at first assessment, analyze its materials, size, angles, weight and other properties. We draw upon our knowledge of the category to which things like this belong, and we figure that this thing is a chair, and that we can sit on it. Later we might think about those properties, but our first thought will always be: that is a chair.

ECS is a more natural way of looking at the world. In a strict sense, a “chair” isn’t a thing, it’s a way of describing certain properties an object might have: flat enough to put a butt on, strong enough hold the average person, made out of a solid material, and so on. The fact that you can sit on it is just one of many properties that this entity holds, and it only becomes a chair once it has enough of these chair-like properties.

This is just one simplified example of how these two systems might treat something like a chair, but the real utility of an entity component system lies in its flexibility. So a chair might not be an entity in itself: it might be made up of four legs, a seat and a back, or of even smaller parts. In this case, the engine can search not for an entity that resembles a chair, but for an arrangement of entities that meets our definition of chair.

What We’re Doing: ECS In Project Artemis

As we mentioned above, our ECS implementation boils down to one very simple idea. This base layer of our engine is about building a system for transforming Data A into Data B with as few assumptions about the data or the processing as possible. At its core it is simply a way of organizing, accessing, and altering data that can be used for any number of different purposes as the situation demands.

Actions carried out by our engine can be as simple as identifying an entity and placing it in the world or as complex as creating animals with realistic behavior and interactions. The only difference is in the number of components contained within a single entity and the number of systems acting on those components.

Moving in Groups: The basic definitional element of our engine is a behavior. In terms of ECS fundamentals a behavior is a system, or a process of changing components from one value to another. That’s really all the engine does: it transforms data by executing behaviors.

Consider one of the most basic behaviors an entity can have: moving. This could apply to animals, vehicles, projectiles, player characters, NPCs or anything else in the world that moves. In terms of ECS, this means altering the “position” components for any entity with the moving behavior. They might move at different speeds or in different directions as defined by components like velocity or vectors, all of which the moving system can pull in order to make its necessary calculations.

On a basic level, this means that a moving system can work on every entity in the world that moves at once. This is a much more efficient way of processing all of this behavior: an object-oriented system might have to look at every object individually and calculate the ways that it is moving, but our system can simply run the moving behavior, calculating for entities at once. It looks at all the relevant components, applies the behavior as necessary, and then changes all the position components based on this system.

How these changed position components impact the world will have to do with which entities they are attached to, because that’s how the engine will know what is actually moving. Crucially, that is not necessary for applying the behavior. The behavior is capable of acting on the data directly rather than concerning itself with which entities those components are attached to. This is a fundamental concept behind data-oriented design.

Growing more complex: Ultimately, our engine will want to do more than simply move different objects around at different speeds. We will also want to do a lot more than we can necessarily account for at the beginning of development, at release, or at any time after that. This is one of the key reasons for approaching development this way: it is built to grow. An entity can have any non-zero number of components, and a component can have any number of systems acting upon it.

For example, we might want to add a physics system to the world. This will also impact position, because things will be getting bumped around and moved as a result. The calculations will be different than the simple “moving” behavior, requiring new components like “weight” as well as old ones like “velocity.” Both of these systems can then interact with “position” to change where entities are in the world, but they can do so independently.

(Clearly, layering systems like these will still require significant tuning work to make sure everything meshes together smoothly, but the fundamental proposition of ECS and data-oriented design should eliminate a lot of the roadblocks that make this incredibly difficult in other situations)

Later on, things might get more complicated. Let’s say we want to add a fire system to the world: we add a new behavior called “burning.” This might apply to most objects that move, but not all, and it does not need to alter the “moving” behavior unless we want it to. We’ll need more components like “flammability” to make this happen, and it will impact components related to weight and appearance. It should be possible to add in these new components and systems without disrupting older ones, and in theory even producing emergent behavior.

Later on, we might want to update the fire system to be more realistic, such as allowing the propagation of fire between entities. We can look at everything that burns in the world and update them with a new system that allows for propagation. Later on, we might want to redesign the fire system from the ground up based on new hardware capabilities or just a changing view of our game world’s needs. We simply remove all systems relating to fire and write new ones. This is still, clearly, a massive undertaking, but the ensuing headaches can be limited by an ECS system where behaviors are processes that can run independently of another. While we may still need to write an entirely new fire system, ideally we do not need to write a new moving system.

Defining the world: In aggregate, these behaviors can serve to define entities. As the old saying goes: if it looks like a duck, swims like a duck and quacks like a duck, it’s probably a duck. The duck is defined by the sum of its behaviors. In this way we can wind up with a much more flexible categorization tool than what an object-oriented system might offer. Ducks might be different sizes and colors, they might move more quickly than others or quack more loudly, but we can recognize them as ducks so long as they share what we have defined as duck-like behaviors. They might also fit into any number of other categories based on their other behaviors and components.

In this way, we end up being able to define the contours of the world in a similar way to object-oriented design, but with an approach that allows for much more flexibility. If a duck is defined by having duck-like properties and not simply by “being a duck,” it leaves that entity with more room for nuance and detail that can be onerous to implement under more rigid systems.

Why We’re Doing It this Way

As explained above, our implementation of ECS is designed to be able to grow over time, allowing us to add more components and more behaviors on top of existing systems. No system would ever be able to eliminate the impact of legacy decisions on future development, but what we can do is design a system that is able to minimize that impact whenever possible. Ideally, that will make it easier to create a scalable and flexible engine over time, capable of handling an ever-increasing array of tasks.

Organization: If the simulation is going to grow continuously, it’s also going to get very big. Our goal is to make an engine capable of handling a massive, complex and interlocking world, and that means we need to build it as efficiently as possible at the most basic level. Because ECS is able to act on component data directly rather than moving through an object system, it is able to access only the data it needs in order to perform the operations it’s performing at any given moment. This allows us to organize the data differently. Instead of a component like “position” being located in a thousand different places attached to a thousand different objects, it can be located in one easy-to-access chunk.

Data can look more like this:

Parallel processing: ECS means that the data and the processing are separate: components make up all the data that defines the world, and systems run on top. Isolating the behavior processing allows for interesting potential when it comes to parallel processing.

In our system, threads do not have specific purposes, they simply execute systems. We can use them to run one system for multiple entities at once, and we can run as many of them as our available cores will allow. This means that if the hardware has more cores, the system can simply run more threads simultaneously. This emphasis means we can use the full potential of both modern CPUs and GPUs for simulation processing, taking advantage of modern graphics APIs like Direct 3D 12 and Vulkan that make it easier and more viable to move tasks to the GPU. With purpose-agnostic processes, it makes it easier to ask the engine to use as much parallel processing power available to it.

A balancing act: Typically, flexibility and efficiency are seen as competing goals. This is one of the potential pitfalls of our assumption-free approach to the engine: assumptions make it possible to introduce the sorts of shortcuts that improve performance, and so fewer assumptions run the risk of becoming a flexible system that does a wide range of things very slowly.

Our system allows those assumptions to all be made later in the process, when the user is making systems that have actual intended functionality. Eventually, we’ll open our engine up to third-party developers and users large and small. This approach means that those developers can have their program make assumptions based on what they actually want to do, not based on what engine developers guessed they might do earlier on.

Building a system like this at the base layer is a daunting task, but the gains in efficiency, flexibility and scalability make it a necessary task for the scope of the project we’re embarking on. We want to build an engine that can last far into the future, and that means starting at the most basic level.