Commits

Michael Ludwig committed 88683a3

Continue heavy refactoring of component build system, and clean up the separation of ComponentData and Component. Almost complete the re-implementation of the ComponentIndex (now ComponentRepository) and EntitySystem with these new changes.

Cleaned up and simplified property configuration and property-factory implementation

Comments (0)

Files changed (28)

src/main/java/com/googlecode/entreri/Component.java

 package com.googlecode.entreri;
 
-import com.googlecode.entreri.property.IndexedDataStore;
-import com.googlecode.entreri.property.Property;
-
 /**
  * <p>
- * ComponentData represents a grouping of reusable and related states that are added
- * to an {@link Entity}. Components are intended to be data storage objects, so
- * their definition should not contain methods for processing or updating (that
- * is the responsibility of a {@link Controller}). Some Components may be
- * defined with an {@link InitParams} annotation, which defines their required
- * arguments when adding a new component to an Entity.
+ * Component represents a grouping of reusable and related states that are added
+ * to an {@link Entity}. The specific state of a component is stored and defined
+ * in {@link ComponentData} implementations. This separation is to support fast
+ * iteration using local memory. All of the component data is packed into
+ * buffers or arrays for cache locality.
  * </p>
  * <p>
- * The behavior or purpose of a ComponentData should be well defined, including its
- * behavior with respect to other Components attached to the same Entity. It may
- * be that to function correctly or--more likely--usefully, related Components
- * will have to be used as well. An example of this might be a transform
- * component and a shape component for rendering.
+ * Component instances represent the identity of the conceptual components,
+ * while instances of ComponentData can be configured to read and write to
+ * specific components. ComponentData's can change which component they
+ * reference multiple times throughout their life time.
  * </p>
- * <p>
- * Each ComponentData class gets a {@link TypeId}, which can be looked up with
- * {@link #getTypedId(Class)}, passing in the desired class type. Because the
- * entity-component design pattern does not follow common object-oriented
- * principles, certain rules are followed when handling ComponentData types in a
- * class hierarchy:
- * <ol>
- * <li>Any abstract type extending ComponentData cannot get a TypeId</li>
- * <li>All concrete classes extending ComponentData get separate TypedIds, even if
- * they extend from the same intermediate classes beneath ComponentData.</li>
- * <li>All intermediate classes in a ComponentData type's hierarchy must be abstract
- * or runtime exceptions will be thrown.</li>
- * </ol>
- * As an example, an abstract component could be Light, with concrete subclasses
- * SpotLight and DirectionLight. SpotLight and DirectionLight would be separate
- * component types as determined by TypeId. Light would not have any TypeId
- * and only serves to consolidate property definition among related component
- * types.
- * </p>
- * <p>
- * Implementations of Components must follow certain rules with respect to their
- * declared fields. For performance reasons, an EntitySystem packs all
- * components of the same type into the same region of memory using the
- * {@link Property} and {@link IndexedDataStore} API. To ensure that Components
- * behave correctly, a type can only declare private or protected Property
- * fields. These fields should be considered "final" from the Components point
- * of view and will be assigned by the EntitySystem. The can be declared final
- * but any assigned value will be overwritten.
- * </p>
- * <p>
- * They can declare any methods they wish to expose the data these properties
- * represent. It is strongly recommended to not expose the Property objects
- * themselves. See {@link #getTypedId(Class)} for the complete contract.
- * </p>
- * <p>
- * ComponentData instances are tied to an index into the IndexedDataStores used by
- * their properties. The index can be fetched by calling {@link #getIndex()}. An
- * instance of ComponentData may have its index changed, effectively changing it to
- * a different "instance". This is most common when using the fast iterators.
- * Because of this, reference equality may not work, instead you should rely on
- * {@link #equals(Object)}.
- * </p>
- * <p>
- * Compone
  * 
  * @author Michael Ludwig
  */
 public final class Component<T extends ComponentData<T>> {
-    private final ComponentIndex<T> owner;
+    private final ComponentRepository<T> owner;
     
-    private int index;
-    private int version;
-    
-    Component(ComponentIndex<T> owner, int index) {
+    int index;
+
+    /**
+     * Create a new Component stored in the given ComponentRepository, at the given
+     * array position within the ComponentRepository.
+     * 
+     * @param owner The ComponentRepository owner
+     * @param index The index within the owner
+     */
+    Component(ComponentRepository<T> owner, int index) {
         this.owner = owner;
         this.index = index;
-        this.version = 0;
     }
-    
+
+    /**
+     * Get a ComponentData instance that can be used to manipulate the state of
+     * this component. This is a convenience for allocating a new ComponentData
+     * instance and assigning it to this component. For tight loops, it is
+     * better to allocate a single ComponentData instance and use its
+     * {@link ComponentData#set(Component) set} method.
+     * 
+     * @return A ComponentData to access this component's state, or null if the
+     *         component is not live
+     */
     public T getData() {
         T data = getEntitySystem().createDataInstance(getTypeId());
         if (data.set(this))
         else
             return null; 
     }
-    
+
+    /**
+     * @return True if the component is still attached to an entity in the
+     *         entity system, or false if it or its entity has been removed
+     */
     public boolean isLive() {
         return index != 0;
     }
-    
+
+    /**
+     * @return True if this component is enabled, or false if it is disabled and
+     *         will appear as though it doesn't exist under default behavior
+     */
     public boolean isEnabled() {
         // if isLive() returns false, index references the 0th index, which
         // just contains garbage
         return owner.isEnabled(index);
     }
-    
+
+    /**
+     * <p>
+     * Set whether or not this component is enabled. If a component is disabled,
+     * default usage will cause it to appear as the component has been removed.
+     * It will not be returned from {@link Entity#get(TypeId)} or be included in
+     * iterator results using {@link ComponentIterator}.
+     * </p>
+     * <p>
+     * Disabling and enabling components can be a more efficient way to simulate
+     * the adding and removing of components, because it does not remove or
+     * require the allocation of new data.
+     * </p>
+     * 
+     * @param enable True if the component is to be enabled
+     */
     public void setEnabled(boolean enable) {
         // if isLive() returns false, index references the 0th index, which
         // just contains garbage so this setter is safe
         owner.setEnabled(index, enable);
     }
-    
+
+    /**
+     * Get the entity that this component is attached to. If the component has
+     * been removed from the entity, or is otherwise not live, this will return
+     * null.
+     * 
+     * @return The owning entity, or null
+     */
     public Entity getEntity() {
         // if isLive() returns false, then the entity index will also be 0,
         // so getEntityByIndex() returns null, which is expected
         return owner.getEntitySystem().getEntityByIndex(entityIndex);
     }
     
+    /**
+     * @return The EntitySystem that created this component
+     */
     public EntitySystem getEntitySystem() {
         return owner.getEntitySystem();
     }
     
+    /**
+     * @return The TypeId of the ComponentData for this Component
+     */
     public TypeId<T> getTypeId() {
         return owner.getTypeId();
     }
     
-    void setIndex(int index) {
-        this.index = index;
-        version++;
-    }
-    
-    int getIndex() {
-        return index;
-    }
-    
-    int getVersion() {
-        return version;
+    /**
+     * @return The ComponentRepository owning this Component
+     */
+    ComponentRepository<T> getRepository() {
+        return owner;
     }
 }

src/main/java/com/googlecode/entreri/ComponentData.java

  * @param <T> The self-referencing type of the ComponentData
  */
 public abstract class ComponentData<T extends ComponentData<T>> {
-    private Component<T> ref;
-    private int version;
+    private int id;
     private int index;
     
-    // this should be considered final, but is assigned in ComponentIndex
+    // this should be considered final, but is assigned in ComponentRepository
     // to simplify implementation constructor requirements.
-    ComponentIndex<T> owner;
+    ComponentRepository<T> owner;
     
     protected ComponentData() { }
     
      *         that is still live
      */
     public final boolean isValid() {
-        return ref != null && ref.getVersion() == version && index != 0;
+        // we have to check the index of the ComponentData because the ComponentRepository
+        // does not make sure the data's indices stay within bounds of the repository arrays
+        return index != 0 && index < owner.getSizeEstimate() && owner.getId(index) == id;
     }
 
     /**
      *             entity system
      */
     public final boolean set(Component<T> component) {
-        if (component.getEntitySystem() != owner.getEntitySystem())
+        // we check repository since it is guaranteed type safe
+        if (component.getRepository() == owner)
             throw new IllegalArgumentException("Component not created by expected EntitySystem");
         
-        this.ref = component;
-        if (ref != null) {
-            // if ref is not live, it's index should have been updated to 0
-            // so the component data will appear invalid as well
-            index = ref.getIndex();
-            version = ref.getVersion();
-        } else {
-            index = 0;
-            version = 0;
-        }
-        
-        return index != 0;
+        return setFast(component.index);
     }
 
     /**
      * A slightly faster method that requires only an index to a component, and
-     * performs no validation.
+     * performs no validation. It also does not look up the component reference
+     * since it assumes it's valid. These are lazily done when needed.
      * 
      * @param componentIndex The index of the component
      * @return True if the index is not 0
      */
     boolean setFast(int componentIndex) {
-        ref = owner.getComponent(componentIndex);
         index = componentIndex;
-        version = (ref == null ? 0 : ref.getVersion());
+        id = owner.getId(index);
         return index != 0;
     }
 }

src/main/java/com/googlecode/entreri/ComponentIndex.java

-/*
- * Entreri, an entity-component framework in Java
- *
- * Copyright (c) 2011, Michael Ludwig
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without modification,
- * are permitted provided that the following conditions are met:
- *
- *     Redistributions of source code must retain the above copyright notice,
- *         this list of conditions and the following disclaimer.
- *     Redistributions in binary form must reproduce the above copyright notice,
- *         this list of conditions and the following disclaimer in the
- *         documentation and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
- * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
- * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
- * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
- * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
- * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- */
-package com.googlecode.entreri;
-
-import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Comparator;
-import java.util.Iterator;
-import java.util.List;
-import java.util.Map;
-import java.util.Map.Entry;
-import java.util.NoSuchElementException;
-
-import com.googlecode.entreri.property.BooleanProperty;
-import com.googlecode.entreri.property.CompactAwareProperty;
-import com.googlecode.entreri.property.IndexedDataStore;
-import com.googlecode.entreri.property.Property;
-import com.googlecode.entreri.property.PropertyFactory;
-
-/**
- * ComponentIndex manages storing all the componentDatas of a specific type for an
- * EntitySystem. It also controls the IndexedDataStore's for the type's set of
- * properties. It is package-private because its details are low-level and
- * complex.
- * 
- * @author Michael Ludwig
- * @param <T> The type of component stored by the index
- */
-final class ComponentIndex<T extends ComponentData<T>> {
-    private final EntitySystem system;
-    private final TypeId<T> type;
-
-    private final ComponentDataFactory<T> factory;
-    
-    // These three arrays have a special value of 0 or null stored in the 0th
-    // index, which allows us to lookup componentDatas or entities when they
-    // normally aren't attached.
-    private int[] entityIndexToComponentIndex;
-    private int[] componentIndexToEntityIndex;
-    private Component<T>[] components;
-    private int componentInsert;
-    
-    private final List<PropertyStore<?>> declaredProperties;
-    private final List<PropertyStore<?>> decoratedProperties;
-    
-    private final BooleanProperty enabledProperty; // this is also contained in decoratedProperties
-     
-    /**
-     * Create a ComponentIndex for the given system, that will store Components
-     * of the given type.
-     * 
-     * @param system The owning system
-     * @param type The type of component
-     * @throws NullPointerException if system or type are null
-     */
-    @SuppressWarnings({ "rawtypes", "unchecked" })
-    public ComponentIndex(EntitySystem system, TypeId<T> type, ComponentDataFactory<T> factory) {
-        if (system == null || type == null)
-            throw new NullPointerException("Arguments cannot be null");
-        
-        this.system = system;
-        this.factory = factory;
-        this.type = type;
-        
-        Map<String, PropertyFactory<?>> propertyFactories = factory.getPropertyFactories();
-        
-        declaredProperties = new ArrayList<PropertyStore<?>>();
-        decoratedProperties = new ArrayList<PropertyStore<?>>(); // empty for now
-        for (Entry<String, PropertyFactory<?>> e: propertyFactories.entrySet()) {
-            PropertyStore store = new PropertyStore(e.getValue(), e.getKey());
-            declaredProperties.add(store);
-        }
-        
-        entityIndexToComponentIndex = new int[1]; // holds default 0 value in 0th index
-        componentIndexToEntityIndex = new int[1]; // holds default 0 value in 0th index
-        components = new Component[1]; // holds default null value in 0th index
-        
-        componentInsert = 1;
-        
-        // Make sure properties' stores hold enough space
-        resizePropertyStores(declaredProperties, 1);
-        
-        // decorate the component data with a boolean property to track enabled status
-        enabledProperty = decorate(BooleanProperty.factory(1));
-    }
-    
-    /**
-     * @return The type of component data stored by this component index
-     */
-    public TypeId<T> getTypeId() {
-        return type;
-    }
-    
-    /**
-     * @return An estimate on the number of components in the index, will not be
-     *         less than the true size
-     */
-    public int getSizeEstimate() {
-        return componentInsert + 1;
-    }
-    
-    /**
-     * @return The owning EntitySystem
-     */
-    public EntitySystem getEntitySystem() {
-        return system;
-    }
-
-    /**
-     * Given the index of a Component (e.g. {@link Component#getIndex()}, return
-     * the index of an entity within the owning system. The returned entity
-     * index can be safely passed to {@link EntitySystem#getEntityByIndex(int)}.
-     * 
-     * @param componentIndex The component index whose owning entity is fetched
-     * @return The index of the entity that has the given component index, or 0
-     *         if the component is not attached
-     */
-    public int getEntityIndex(int componentIndex) {
-        return componentIndexToEntityIndex[componentIndex];
-    }
-
-    /**
-     * Given the index of an entity (e.g. {@link Entity#index}), return the
-     * index of the attached component of this ComponentIndex's type. The
-     * returned component index can be used in {@link #getComponent(int)} and
-     * related methods.
-     * 
-     * @param entityIndex The entity index to look up
-     * @return The index of the attached component, or 0 if the entity does not
-     *         have a component of this type attached
-     */
-    public int getComponentIndex(int entityIndex) {
-        return entityIndexToComponentIndex[entityIndex];
-    }
-
-    /**
-     * Ensure that this ComponentIndex has enough internal space to hold its
-     * entity-to-component mapping for the given number of entities.
-     * 
-     * @param numEntities The new number of entities
-     */
-    public void expandEntityIndex(int numEntities) {
-        if (entityIndexToComponentIndex.length < numEntities) {
-            entityIndexToComponentIndex = Arrays.copyOf(entityIndexToComponentIndex, (int) (numEntities * 1.5f) + 1);
-        }
-    }
-
-    /**
-     * @param componentIndex The component to look up
-     * @return True if the component at componentIndex is enabled
-     */
-    public boolean isEnabled(int componentIndex) {
-        return enabledProperty.get(componentIndex, 0);
-    }
-
-    /**
-     * Set whether or not the component at <tt>componentIndex</tt> is enabled.
-     * 
-     * @param componentIndex The component index
-     * @param enabled True if the component is enabled
-     */
-    public void setEnabled(int componentIndex, boolean enabled) {
-        enabledProperty.set(enabled, componentIndex, 0);
-    }
-    
-    /*
-     * As expandEntityIndex() but expands all related component data and arrays
-     * to hold the number of components.
-     */
-    private void expandComponentIndex(int numComponents) {
-        if (numComponents < components.length)
-            return;
-
-        int size = (int) (numComponents * 1.5f) + 1;
-        
-        // Expand the indexed data stores for the properties
-        resizePropertyStores(declaredProperties, size);
-        resizePropertyStores(decoratedProperties, size);
-        
-        // Expand the canonical component array
-        components = Arrays.copyOf(components, size);
-        
-        // Expand the component index
-        componentIndexToEntityIndex = Arrays.copyOf(componentIndexToEntityIndex, size);
-    }
-
-    /*
-     * Convenience to create a new data store for each property with the given
-     * size, copy the old data over, and assign it back to the property.
-     */
-    private void resizePropertyStores(List<PropertyStore<?>> properties, int size) {
-        int ct = properties.size();
-        for (int i = 0; i < ct; i++) {
-            IndexedDataStore oldStore = properties.get(i).property.getDataStore();
-            IndexedDataStore newStore = oldStore.create(size);
-            oldStore.copy(0, Math.min(oldStore.size(), size), newStore, 0);
-            properties.get(i).property.setDataStore(newStore);
-        }
-    }
-    
-    /**
-     * @param componentIndex The component index whose component is fetched
-     * @return The component reference at the given index, may be null
-     */
-    public Component<T> getComponent(int componentIndex) {
-        return components[componentIndex];
-    }
-
-    /**
-     * Create a new component and attach to it the entity at the given entity
-     * index, the new component will have its values copied from the existing
-     * template.
-     * 
-     * @param entityIndex The entity index which the component is attached to
-     * @param fromTemplate A template to assign values to the new component
-     * @return A new component of type T
-     * @throws NullPointerException if fromTemplate is null
-     * @throws IllegalArgumentException if the template was not created by this
-     *             index
-     * @throws IllegalStateException if the template is not live
-     */
-    @SuppressWarnings({ "unchecked", "rawtypes" })
-    public Component<T> addComponent(int entityIndex, Component<T> fromTemplate) {
-        if (fromTemplate.getEntitySystem() != getEntitySystem())
-            throw new IllegalArgumentException("Component not owned by expected EntitySystem");
-        if (fromTemplate.getTypeId().equals(type))
-            throw new IllegalArgumentException("Component not of expected type, expected: " + type + ", but was: " + type);
-        if (!fromTemplate.isLive())
-            throw new IllegalStateException("Template component is not live");
-        
-        Component<T> instance = addComponent(entityIndex);
-        for (int i = 0; i < declaredProperties.size(); i++) {
-            PropertyStore store = declaredProperties.get(i);
-            store.clone(fromTemplate.getIndex(), store.property, instance.getIndex());
-        }
-        
-        // fire add-event listener after cloning is completed
-        system.getControllerManager().fireComponentAdd(instance);
-        return instance;
-    }
-
-    /**
-     * Create a new component and attach to it the entity at the given entity
-     * index. The component will have the default state as specified by its
-     * properties.
-     * 
-     * @param entityIndex The entity index which the component is attached to
-     * @param initParams The var-args of parameters that must match the
-     *            InitParams annotation of the type
-     * @return A new component of type T
-     * @throws IllegalArgumentException if initParams is incorrect
-     */
-    public Component<T> addComponent(int entityIndex) {
-        Component<T> instance = allocateComponent(entityIndex);
-        
-        // fire add-event listener after initialization is completed
-        system.getControllerManager().fireComponentAdd(instance);
-        return instance;
-    }
-    
-    /*
-     * Allocate and store a new component, but don't initialize it yet.
-     */
-    private Component<T> allocateComponent(int entityIndex) {
-        if (entityIndexToComponentIndex[entityIndex] != 0)
-            removeComponent(entityIndex);
-        
-        int componentIndex = componentInsert++;
-        if (componentIndex >= components.length)
-            expandComponentIndex(componentIndex + 1);
-
-        Component<T> instance = new Component<T>(this, componentIndex);
-        components[componentIndex] = instance;
-        componentIndexToEntityIndex[componentIndex] = entityIndex;
-        entityIndexToComponentIndex[entityIndex] = componentIndex;
-
-        // Set default value for declared and decorated properties,
-        // this is needed because we might be overwriting a previously removed
-        // component, or the factory might be doing something tricky
-        for (int i = 0; i < declaredProperties.size(); i++) {
-            declaredProperties.get(i).setValue(componentIndex);
-        }
-        
-        for (int i = 0; i < decoratedProperties.size(); i++) {
-            decoratedProperties.get(i).setValue(componentIndex);
-        }
-        
-        return instance;
-    }
-
-    /**
-     * Create a new ComponentData of type T that can be used to view components
-     * in this index.
-     * 
-     * @return A new data instance
-     */
-    public T createDataInstance() {
-        // create a new instance from the factory - it will be completely detached
-        T t = factory.createInstance();
-        
-        // attach it to this data index, at the 0th index
-        //  - at this point the ComponentData's owner should be considered final
-        t.owner = this;
-        
-        // assign all property values
-        for (int i = 0; i < declaredProperties.size(); i++) {
-            PropertyStore<?> p = declaredProperties.get(i);
-            factory.setProperty(t, p.key, p.property);
-        }
-        
-        t.set(null);
-        return t;
-    }
-
-    /**
-     * Detach or remove any component of this index's type from the entity with
-     * the given index. True is returned if a component was removed, or false
-     * otherwise.
-     * 
-     * @param entityIndex The entity's index whose component is removed
-     * @return True if a component was removed
-     */
-    public boolean removeComponent(int entityIndex) {
-        int componentIndex = entityIndexToComponentIndex[entityIndex];
-
-        // This code works even if componentIndex is 0
-        Component<T> oldComponent = components[componentIndex];
-        if (oldComponent != null) {
-            // perform component clean up before data is invalidated
-            system.getControllerManager().fireComponentRemove(oldComponent);
-            oldComponent.setIndex(0);
-        }
-
-        components[componentIndex] = null;
-        entityIndexToComponentIndex[entityIndex] = 0; // entity does not have component
-        componentIndexToEntityIndex[componentIndex] = 0; // component does not have entity
-        
-        return oldComponent != null;
-    }
-
-    /*
-     * Update all component data in the list of properties. If possible the data
-     * store in swap is reused.
-     */
-    private void update(List<PropertyStore<?>> properties, Component<T>[] newToOldMap) {
-        for (int i = 0; i < properties.size(); i++) {
-            PropertyStore<?> p = properties.get(i);
-            IndexedDataStore origStore = p.property.getDataStore();
-            
-            p.property.setDataStore(update(origStore, p.swap, newToOldMap));
-            p.swap = origStore;
-        }
-    }
-
-    /*
-     * Update all component data in src to be in dst by shuffling it to match
-     * newToOldMap.
-     */
-    private IndexedDataStore update(IndexedDataStore src, IndexedDataStore dst, 
-                                    Component<T>[] newToOldMap) {
-        int dstSize = newToOldMap.length;
-        
-        if (dst == null || dst.size() < dstSize)
-            dst = src.create(dstSize);
-        
-        int i;
-        int lastIndex = -1;
-        int copyIndexNew = -1;
-        int copyIndexOld = -1;
-        for (i = 1; i < componentInsert; i++) {
-            if (newToOldMap[i] == null) {
-                // we've hit the end of existing componentDatas, so break
-                break;
-            }
-            
-            if (newToOldMap[i].getIndex() != lastIndex + 1) {
-                // we are not in a contiguous section
-                if (copyIndexOld >= 0) {
-                    // we have to copy over the last section
-                    src.copy(copyIndexOld, (i - copyIndexNew), dst, copyIndexNew);
-                }
-                
-                // set the copy indices
-                copyIndexNew = i;
-                copyIndexOld = newToOldMap[i].getIndex();
-            }
-            lastIndex = newToOldMap[i].getIndex();
-        }
-        
-        if (copyIndexOld >= 0) {
-            // final copy
-            src.copy(copyIndexOld, (i - copyIndexNew), dst, copyIndexNew);
-        }
-
-        return dst;
-    }
-    
-    private void notifyCompactAwareProperties(List<PropertyStore<?>> props) {
-        PropertyStore<?> p;
-        for (int i = 0; i < props.size(); i++) {
-            p = props.get(i);
-            if (p.property instanceof CompactAwareProperty)
-                ((CompactAwareProperty) p.property).onCompactComplete();
-        }
-    }
-
-    /**
-     * <p>
-     * Compact the data of this ComponentIndex to account for removals and
-     * additions to the index. This will ensure that all active componentDatas are
-     * packed into the underlying arrays, and that they will be accessed in the
-     * same order as iterating over the entities directly.
-     * </p>
-     * <p>
-     * The map from old to new entity index must be used to properly update the
-     * component index's data so that the system is kept in sync.
-     * </p>
-     * 
-     * @param entityOldToNewMap A map from old entity index to new index
-     * @param numEntities The number of entities that are in the system
-     */
-    public void compact(int[] entityOldToNewMap, int numEntities) {
-        // First sort the canonical componentDatas array
-        Arrays.sort(components, 1, componentInsert, new Comparator<Component<T>>() {
-            @Override
-            public int compare(Component<T> o1, Component<T> o2) {
-                if (o1 != null && o2 != null)
-                    return componentIndexToEntityIndex[o1.getIndex()] - componentIndexToEntityIndex[o2.getIndex()];
-                else if (o1 != null)
-                    return -1; // push null o2 to end of array
-                else if (o2 != null)
-                    return 1; // push null o1 to end of array
-                else
-                    return 0; // both null so they are "equal"
-            }
-        });
-        
-        // Update all of the property stores to match up with the componentDatas new positions
-        update(declaredProperties, components);
-        update(decoratedProperties, components);
-        
-        // Repair the componentToEntityIndex and the component.index values
-        componentInsert = 1;
-        int[] newComponentIndex = new int[components.length];
-        for (int i = 1; i < components.length; i++) {
-            if (components[i] != null) {
-                newComponentIndex[i] = entityOldToNewMap[componentIndexToEntityIndex[components[i].getIndex()]];
-                components[i].setIndex(i);
-                componentInsert = i + 1;
-            }
-        }
-        componentIndexToEntityIndex = newComponentIndex;
-        
-        // Possibly compact the component data
-        if (componentInsert < .6f * components.length) {
-            int newSize = (int) (1.2f * componentInsert) + 1;
-            components = Arrays.copyOf(components, newSize);
-            componentIndexToEntityIndex = Arrays.copyOf(componentIndexToEntityIndex, newSize);
-            resizePropertyStores(declaredProperties, newSize);
-            resizePropertyStores(decoratedProperties, newSize);
-        }
-        
-        // Repair entityIndexToComponentIndex - and possible shrink the index
-        // based on the number of packed entities
-        if (numEntities < .6f * entityIndexToComponentIndex.length)
-            entityIndexToComponentIndex = new int[(int) (1.2f * numEntities) + 1];
-        else
-            Arrays.fill(entityIndexToComponentIndex, 0);
-        
-        for (int i = 1; i < componentInsert; i++)
-            entityIndexToComponentIndex[componentIndexToEntityIndex[i]] = i;
-        
-        notifyCompactAwareProperties(declaredProperties);
-        notifyCompactAwareProperties(decoratedProperties);
-    }
-
-    /**
-     * @return An iterator over the components in the index. The iterator's
-     *         remove() detaches the component from the entity
-     */
-    public Iterator<Component<T>> iterator() {
-        return new ComponentIterator();
-    }
-
-    /**
-     * Decorate the type information of this ComponentIndex to add a property
-     * created by the given factory. The returned property will have default
-     * data assigned for each current Component in the index, and will have the
-     * default value assigned for each new Component. Decorators can then access
-     * the returned property to manipulate the decorated component data.
-     * 
-     * @param <P> The type of property created
-     * @param factory The factory that will create a unique Property instance
-     *            associated with the decorated property and this index
-     * @return The property decorated onto the type of the index
-     */
-    public <P extends Property> P decorate(PropertyFactory<P> factory) {
-        int size = (declaredProperties.isEmpty() ? componentInsert
-                                                 : declaredProperties.get(0).property.getDataStore().size());
-        
-        PropertyStore<P> pstore = new PropertyStore<P>(factory, "decorated");
-
-        // Set values from factory to all component slots
-        IndexedDataStore newStore = pstore.property.getDataStore().create(size);
-        pstore.property.setDataStore(newStore);
-        for (int i = 1; i < size; i++) {
-            pstore.setValue(i);
-        }
-        
-        decoratedProperties.add(pstore);
-        return pstore.property;
-    }
-
-    /**
-     * Remove the given property from the set of decorated properties on this
-     * index's type. If the property is invalid or not a decorated property for
-     * the index, this does nothing.
-     * 
-     * @param p The property to remove
-     */
-    public void undecorate(Property p) {
-        Iterator<PropertyStore<?>> it = decoratedProperties.iterator();
-        while(it.hasNext()) {
-            if (it.next().property == p) {
-                it.remove();
-                break;
-            }
-        }
-    }
-    
-    /*
-     * An iterator implementation over the canonical componentDatas of the index.
-     */
-    private class ComponentIterator implements Iterator<Component<T>> {
-        private int index;
-        private boolean advanced;
-        
-        public ComponentIterator() {
-            index = 0;
-            advanced = false;
-        }
-        
-        @Override
-        public boolean hasNext() {
-            if (!advanced)
-                advance();
-            return index < componentInsert;
-        }
-
-        @Override
-        public Component<T> next() {
-            if (!hasNext())
-                throw new NoSuchElementException();
-            advanced = false;
-            return components[index];
-        }
-
-        @Override
-        public void remove() {
-            if (advanced || index == 0)
-                throw new IllegalStateException("Must call next() before remove()");
-            if (componentIndexToEntityIndex[index] == 0)
-                throw new IllegalStateException("ComponentData already removed");
-            removeComponent(componentIndexToEntityIndex[index]);
-        }
-        
-        private void advance() {
-            index++; // always advance at least 1
-            while(index < components.length && componentIndexToEntityIndex[index] == 0) {
-                index++;
-            }
-            advanced = true;
-        }
-    }
-
-    /*
-     * Type wrapping a key, property, and factory, as well as an auxiliary data
-     * store for compaction.
-     */
-    private static class PropertyStore<P extends Property> {
-        final String key;
-        final P property;
-        final PropertyFactory<P> creator;
-        IndexedDataStore swap; // may be null
-        
-        
-        public PropertyStore(PropertyFactory<P> creator, String key) {
-            this.creator = creator;
-            this.key = key;
-            property = creator.create();
-        }
-        
-        private void clone(int srcIndex, P dst, int dstIndex) {
-            creator.clone(property, srcIndex, dst, dstIndex);
-        }
-        
-        private void setValue(int index) {
-            creator.setValue(property, index);
-        }
-    }
-}

src/main/java/com/googlecode/entreri/ComponentIterator.java

     
     private final EntitySystem system;
     
+    // FIXME: to avoid pulling in the Entity objects that this iterates over,
+    // I think it would be best if we had an index into each component store.
+    // We can then validate quickly the CDs by checking the componentToEntity index.
     private int index;
     // FIXME: do we also store the shortest component type?
     // FIXME: should I just use the iterator's over component type?
     // FIXME: if not, can I remove them?
     
+    // For some time I had thought that it might be faster to walk over all component types
+    // at the same time, but that actually won't work because each type's index
+    // will not have the same sorted order (unless a compact() is performed).
+    // Since we can't assume that, we have to use the method where the shortest
+    // component type is used and the entity-component-index is checked for all
+    // other types
+    
     public ComponentIterator(EntitySystem system) {
         if (system == null)
             throw new NullPointerException("System cannot be null");

src/main/java/com/googlecode/entreri/ComponentRepository.java

+/*
+ * Entreri, an entity-component framework in Java
+ *
+ * Copyright (c) 2011, Michael Ludwig
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without modification,
+ * are permitted provided that the following conditions are met:
+ *
+ *     Redistributions of source code must retain the above copyright notice,
+ *         this list of conditions and the following disclaimer.
+ *     Redistributions in binary form must reproduce the above copyright notice,
+ *         this list of conditions and the following disclaimer in the
+ *         documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
+ * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+package com.googlecode.entreri;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.NoSuchElementException;
+
+import com.googlecode.entreri.property.BooleanProperty;
+import com.googlecode.entreri.property.CompactAwareProperty;
+import com.googlecode.entreri.property.IndexedDataStore;
+import com.googlecode.entreri.property.IntProperty;
+import com.googlecode.entreri.property.Property;
+import com.googlecode.entreri.property.PropertyFactory;
+
+/**
+ * ComponentRepository manages storing all the componentDatas of a specific type for an
+ * EntitySystem. It also controls the IndexedDataStore's for the type's set of
+ * properties. It is package-private because its details are low-level and
+ * complex.
+ * 
+ * @author Michael Ludwig
+ * @param <T> The type of component stored by the index
+ */
+final class ComponentRepository<T extends ComponentData<T>> {
+    private final EntitySystem system;
+    private final TypeId<T> type;
+
+    private final ComponentDataFactory<T> factory;
+    
+    // These three arrays have a special value of 0 or null stored in the 0th
+    // index, which allows us to lookup componentDatas or entities when they
+    // normally aren't attached.
+    private int[] entityIndexToComponentRepository;
+    private int[] componentIndexToEntityIndex;
+    private Component<T>[] components;
+    private int componentInsert;
+    
+    private final List<PropertyStore<?>> declaredProperties;
+    private final List<PropertyStore<?>> decoratedProperties;
+    
+    private final BooleanProperty enabledProperty; // this is also contained in decoratedProperties
+    private final IntProperty componentIdProperty; // this is contained in decoratedProperties
+    private int idSeq;
+     
+    /**
+     * Create a ComponentRepository for the given system, that will store Components
+     * of the given type.
+     * 
+     * @param system The owning system
+     * @param type The type of component
+     * @throws NullPointerException if system or type are null
+     */
+    @SuppressWarnings({ "rawtypes", "unchecked" })
+    public ComponentRepository(EntitySystem system, TypeId<T> type, ComponentDataFactory<T> factory) {
+        if (system == null || type == null)
+            throw new NullPointerException("Arguments cannot be null");
+        
+        this.system = system;
+        this.factory = factory;
+        this.type = type;
+        
+        Map<String, PropertyFactory<?>> propertyFactories = factory.getPropertyFactories();
+        
+        declaredProperties = new ArrayList<PropertyStore<?>>();
+        decoratedProperties = new ArrayList<PropertyStore<?>>(); // empty for now
+        for (Entry<String, PropertyFactory<?>> e: propertyFactories.entrySet()) {
+            PropertyStore store = new PropertyStore(e.getValue(), e.getKey());
+            declaredProperties.add(store);
+        }
+        
+        entityIndexToComponentRepository = new int[1]; // holds default 0 value in 0th index
+        componentIndexToEntityIndex = new int[1]; // holds default 0 value in 0th index
+        components = new Component[1]; // holds default null value in 0th index
+        
+        componentInsert = 1;
+        
+        // Make sure properties' stores hold enough space
+        resizePropertyStores(declaredProperties, 1);
+        
+        // decorate the component data with a boolean property to track enabled status
+        enabledProperty = decorate(BooleanProperty.factory(1, true));
+        componentIdProperty = decorate(IntProperty.factory(1)); // we'll not assign a default value, since we change the id each time
+        idSeq = 1; // start at 1, just like entity id sequences
+    }
+    
+    /**
+     * @return The type of component data stored by this component index
+     */
+    public TypeId<T> getTypeId() {
+        return type;
+    }
+    
+    /**
+     * @return An estimate on the number of components in the index, will not be
+     *         less than the true size
+     */
+    public int getSizeEstimate() {
+        return componentInsert + 1;
+    }
+    
+    /**
+     * @return The owning EntitySystem
+     */
+    public EntitySystem getEntitySystem() {
+        return system;
+    }
+
+    /**
+     * Given the index of a Component (e.g. {@link Component#getIndex()}, return
+     * the index of an entity within the owning system. The returned entity
+     * index can be safely passed to {@link EntitySystem#getEntityByIndex(int)}.
+     * 
+     * @param componentIndex The component index whose owning entity is fetched
+     * @return The index of the entity that has the given component index, or 0
+     *         if the component is not attached
+     */
+    public int getEntityIndex(int componentIndex) {
+        return componentIndexToEntityIndex[componentIndex];
+    }
+
+    /**
+     * Given the index of an entity (e.g. {@link Entity#index}), return the
+     * index of the attached component of this ComponentRepository's type. The
+     * returned component index can be used in {@link #getComponent(int)} and
+     * related methods.
+     * 
+     * @param entityIndex The entity index to look up
+     * @return The index of the attached component, or 0 if the entity does not
+     *         have a component of this type attached
+     */
+    public int getComponentRepository(int entityIndex) {
+        return entityIndexToComponentRepository[entityIndex];
+    }
+
+    /**
+     * Ensure that this ComponentRepository has enough internal space to hold its
+     * entity-to-component mapping for the given number of entities.
+     * 
+     * @param numEntities The new number of entities
+     */
+    public void expandEntityIndex(int numEntities) {
+        if (entityIndexToComponentRepository.length < numEntities) {
+            entityIndexToComponentRepository = Arrays.copyOf(entityIndexToComponentRepository, (int) (numEntities * 1.5f) + 1);
+        }
+    }
+
+    /**
+     * @param componentIndex The component to look up
+     * @return True if the component at componentIndex is enabled
+     */
+    public boolean isEnabled(int componentIndex) {
+        return enabledProperty.get(componentIndex, 0);
+    }
+
+    /**
+     * Set whether or not the component at <tt>componentIndex</tt> is enabled.
+     * 
+     * @param componentIndex The component index
+     * @param enabled True if the component is enabled
+     */
+    public void setEnabled(int componentIndex, boolean enabled) {
+        enabledProperty.set(enabled, componentIndex, 0);
+    }
+    
+    public int getId(int componentIndex) {
+        return componentIdProperty.get(componentIndex, 0);
+    }
+    
+    /*
+     * As expandEntityIndex() but expands all related component data and arrays
+     * to hold the number of components.
+     */
+    private void expandComponentRepository(int numComponents) {
+        if (numComponents < components.length)
+            return;
+
+        int size = (int) (numComponents * 1.5f) + 1;
+        
+        // Expand the indexed data stores for the properties
+        resizePropertyStores(declaredProperties, size);
+        resizePropertyStores(decoratedProperties, size);
+        
+        // Expand the canonical component array
+        components = Arrays.copyOf(components, size);
+        
+        // Expand the component index
+        componentIndexToEntityIndex = Arrays.copyOf(componentIndexToEntityIndex, size);
+    }
+
+    /*
+     * Convenience to create a new data store for each property with the given
+     * size, copy the old data over, and assign it back to the property.
+     */
+    private void resizePropertyStores(List<PropertyStore<?>> properties, int size) {
+        int ct = properties.size();
+        for (int i = 0; i < ct; i++) {
+            IndexedDataStore oldStore = properties.get(i).property.getDataStore();
+            IndexedDataStore newStore = oldStore.create(size);
+            oldStore.copy(0, Math.min(oldStore.size(), size), newStore, 0);
+            properties.get(i).property.setDataStore(newStore);
+        }
+    }
+    
+    /**
+     * @param componentIndex The component index whose component is fetched
+     * @return The component reference at the given index, may be null
+     */
+    public Component<T> getComponent(int componentIndex) {
+        return components[componentIndex];
+    }
+
+    /**
+     * Create a new component and attach to it the entity at the given entity
+     * index, the new component will have its values copied from the existing
+     * template.
+     * 
+     * @param entityIndex The entity index which the component is attached to
+     * @param fromTemplate A template to assign values to the new component
+     * @return A new component of type T
+     * @throws NullPointerException if fromTemplate is null
+     * @throws IllegalArgumentException if the template was not created by this
+     *             index
+     * @throws IllegalStateException if the template is not live
+     */
+    @SuppressWarnings({ "unchecked", "rawtypes" })
+    public Component<T> addComponent(int entityIndex, Component<T> fromTemplate) {
+        if (fromTemplate.getEntitySystem() != getEntitySystem())
+            throw new IllegalArgumentException("Component not owned by expected EntitySystem");
+        if (fromTemplate.getTypeId().equals(type))
+            throw new IllegalArgumentException("Component not of expected type, expected: " + type + ", but was: " + type);
+        if (!fromTemplate.isLive())
+            throw new IllegalStateException("Template component is not live");
+        
+        Component<T> instance = addComponent(entityIndex);
+        for (int i = 0; i < declaredProperties.size(); i++) {
+            PropertyStore store = declaredProperties.get(i);
+            store.clone(fromTemplate.index, store.property, instance.index);
+        }
+        
+        // fire add-event listener after cloning is completed
+        system.getControllerManager().fireComponentAdd(instance);
+        return instance;
+    }
+
+    /**
+     * Create a new component and attach to it the entity at the given entity
+     * index. The component will have the default state as specified by its
+     * properties.
+     * 
+     * @param entityIndex The entity index which the component is attached to
+     * @param initParams The var-args of parameters that must match the
+     *            InitParams annotation of the type
+     * @return A new component of type T
+     * @throws IllegalArgumentException if initParams is incorrect
+     */
+    public Component<T> addComponent(int entityIndex) {
+        Component<T> instance = allocateComponent(entityIndex);
+        
+        // fire add-event listener after initialization is completed
+        system.getControllerManager().fireComponentAdd(instance);
+        return instance;
+    }
+    
+    /*
+     * Allocate and store a new component, but don't initialize it yet.
+     */
+    private Component<T> allocateComponent(int entityIndex) {
+        if (entityIndexToComponentRepository[entityIndex] != 0)
+            removeComponent(entityIndex);
+        
+        int componentIndex = componentInsert++;
+        if (componentIndex >= components.length)
+            expandComponentRepository(componentIndex + 1);
+
+        Component<T> instance = new Component<T>(this, componentIndex);
+        components[componentIndex] = instance;
+        componentIndexToEntityIndex[componentIndex] = entityIndex;
+        entityIndexToComponentRepository[entityIndex] = componentIndex;
+
+        // Set default value for declared and decorated properties,
+        // this is needed because we might be overwriting a previously removed
+        // component, or the factory might be doing something tricky
+        for (int i = 0; i < declaredProperties.size(); i++) {
+            declaredProperties.get(i).setValue(componentIndex);
+        }
+        
+        for (int i = 0; i < decoratedProperties.size(); i++) {
+            decoratedProperties.get(i).setValue(componentIndex);
+        }
+        
+        // although there could be a custom PropertyFactory for setting the id,
+        // it's easier to assign a new id here
+        componentIdProperty.set(idSeq++, componentIndex, 0);
+        
+        return instance;
+    }
+
+    /**
+     * Create a new ComponentData of type T that can be used to view components
+     * in this index.
+     * 
+     * @return A new data instance
+     */
+    public T createDataInstance() {
+        // create a new instance from the factory - it will be completely detached
+        T t = factory.createInstance();
+        
+        // attach it to this data index, at the 0th index
+        //  - at this point the ComponentData's owner should be considered final
+        t.owner = this;
+        
+        // assign all property values
+        for (int i = 0; i < declaredProperties.size(); i++) {
+            PropertyStore<?> p = declaredProperties.get(i);
+            factory.setProperty(t, p.key, p.property);
+        }
+        
+        t.set(null);
+        return t;
+    }
+
+    /**
+     * Detach or remove any component of this index's type from the entity with
+     * the given index. True is returned if a component was removed, or false
+     * otherwise.
+     * 
+     * @param entityIndex The entity's index whose component is removed
+     * @return True if a component was removed
+     */
+    public boolean removeComponent(int entityIndex) {
+        int componentIndex = entityIndexToComponentRepository[entityIndex];
+
+        // This code works even if componentIndex is 0
+        Component<T> oldComponent = components[componentIndex];
+        if (oldComponent != null) {
+            // perform component clean up before data is invalidated
+            system.getControllerManager().fireComponentRemove(oldComponent);
+            oldComponent.index = 0;
+        }
+
+        components[componentIndex] = null;
+        entityIndexToComponentRepository[entityIndex] = 0; // entity does not have component
+        componentIndexToEntityIndex[componentIndex] = 0; // component does not have entity
+        
+        return oldComponent != null;
+    }
+
+    /*
+     * Update all component data in the list of properties. If possible the data
+     * store in swap is reused.
+     */
+    private void update(List<PropertyStore<?>> properties, Component<T>[] newToOldMap) {
+        for (int i = 0; i < properties.size(); i++) {
+            PropertyStore<?> p = properties.get(i);
+            IndexedDataStore origStore = p.property.getDataStore();
+            
+            p.property.setDataStore(update(origStore, p.swap, newToOldMap));
+            p.swap = origStore;
+        }
+    }
+
+    /*
+     * Update all component data in src to be in dst by shuffling it to match
+     * newToOldMap.
+     */
+    private IndexedDataStore update(IndexedDataStore src, IndexedDataStore dst, 
+                                    Component<T>[] newToOldMap) {
+        int dstSize = newToOldMap.length;
+        
+        if (dst == null || dst.size() < dstSize)
+            dst = src.create(dstSize);
+        
+        int i;
+        int lastIndex = -1;
+        int copyIndexNew = -1;
+        int copyIndexOld = -1;
+        for (i = 1; i < componentInsert; i++) {
+            if (newToOldMap[i] == null) {
+                // we've hit the end of existing componentDatas, so break
+                break;
+            }
+            
+            if (newToOldMap[i].index != lastIndex + 1) {
+                // we are not in a contiguous section
+                if (copyIndexOld >= 0) {
+                    // we have to copy over the last section
+                    src.copy(copyIndexOld, (i - copyIndexNew), dst, copyIndexNew);
+                }
+                
+                // set the copy indices
+                copyIndexNew = i;
+                copyIndexOld = newToOldMap[i].index;
+            }
+            lastIndex = newToOldMap[i].index;
+        }
+        
+        if (copyIndexOld >= 0) {
+            // final copy
+            src.copy(copyIndexOld, (i - copyIndexNew), dst, copyIndexNew);
+        }
+
+        return dst;
+    }
+    
+    private void notifyCompactAwareProperties(List<PropertyStore<?>> props) {
+        PropertyStore<?> p;
+        for (int i = 0; i < props.size(); i++) {
+            p = props.get(i);
+            if (p.property instanceof CompactAwareProperty)
+                ((CompactAwareProperty) p.property).onCompactComplete();
+        }
+    }
+
+    /**
+     * <p>
+     * Compact the data of this ComponentRepository to account for removals and
+     * additions to the index. This will ensure that all active componentDatas are
+     * packed into the underlying arrays, and that they will be accessed in the
+     * same order as iterating over the entities directly.
+     * </p>
+     * <p>
+     * The map from old to new entity index must be used to properly update the
+     * component index's data so that the system is kept in sync.
+     * </p>
+     * 
+     * @param entityOldToNewMap A map from old entity index to new index
+     * @param numEntities The number of entities that are in the system
+     */
+    public void compact(int[] entityOldToNewMap, int numEntities) {
+        // First sort the canonical componentDatas array
+        Arrays.sort(components, 1, componentInsert, new Comparator<Component<T>>() {
+            @Override
+            public int compare(Component<T> o1, Component<T> o2) {
+                if (o1 != null && o2 != null)
+                    return componentIndexToEntityIndex[o1.index] - componentIndexToEntityIndex[o2.index];
+                else if (o1 != null)
+                    return -1; // push null o2 to end of array
+                else if (o2 != null)
+                    return 1; // push null o1 to end of array
+                else
+                    return 0; // both null so they are "equal"
+            }
+        });
+        
+        // Update all of the property stores to match up with the componentDatas new positions
+        update(declaredProperties, components);
+        update(decoratedProperties, components);
+        
+        // Repair the componentToEntityIndex and the component.index values
+        componentInsert = 1;
+        int[] newComponentRepository = new int[components.length];
+        for (int i = 1; i < components.length; i++) {
+            if (components[i] != null) {
+                newComponentRepository[i] = entityOldToNewMap[componentIndexToEntityIndex[components[i].index]];
+                components[i].index = i;
+                componentInsert = i + 1;
+            }
+        }
+        componentIndexToEntityIndex = newComponentRepository;
+        
+        // Possibly compact the component data
+        if (componentInsert < .6f * components.length) {
+            int newSize = (int) (1.2f * componentInsert) + 1;
+            components = Arrays.copyOf(components, newSize);
+            componentIndexToEntityIndex = Arrays.copyOf(componentIndexToEntityIndex, newSize);
+            resizePropertyStores(declaredProperties, newSize);
+            resizePropertyStores(decoratedProperties, newSize);
+        }
+        
+        // Repair entityIndexToComponentRepository - and possible shrink the index
+        // based on the number of packed entities
+        if (numEntities < .6f * entityIndexToComponentRepository.length)
+            entityIndexToComponentRepository = new int[(int) (1.2f * numEntities) + 1];
+        else
+            Arrays.fill(entityIndexToComponentRepository, 0);
+        
+        for (int i = 1; i < componentInsert; i++)
+            entityIndexToComponentRepository[componentIndexToEntityIndex[i]] = i;
+        
+        notifyCompactAwareProperties(declaredProperties);
+        notifyCompactAwareProperties(decoratedProperties);
+    }
+
+    /**
+     * @return An iterator over the components in the index. The iterator's
+     *         remove() detaches the component from the entity
+     */
+    public Iterator<Component<T>> iterator() {
+        return new ComponentIterator();
+    }
+
+    /**
+     * Decorate the type information of this ComponentRepository to add a property
+     * created by the given factory. The returned property will have default
+     * data assigned for each current Component in the index, and will have the
+     * default value assigned for each new Component. Decorators can then access
+     * the returned property to manipulate the decorated component data.
+     * 
+     * @param <P> The type of property created
+     * @param factory The factory that will create a unique Property instance
+     *            associated with the decorated property and this index
+     * @return The property decorated onto the type of the index
+     */
+    public <P extends Property> P decorate(PropertyFactory<P> factory) {
+        int size = (declaredProperties.isEmpty() ? componentInsert
+                                                 : declaredProperties.get(0).property.getDataStore().size());
+        
+        PropertyStore<P> pstore = new PropertyStore<P>(factory, "decorated");
+
+        // Set values from factory to all component slots
+        IndexedDataStore newStore = pstore.property.getDataStore().create(size);
+        pstore.property.setDataStore(newStore);
+        for (int i = 1; i < size; i++) {
+            pstore.setValue(i);
+        }
+        
+        decoratedProperties.add(pstore);
+        return pstore.property;
+    }
+
+    /**
+     * Remove the given property from the set of decorated properties on this
+     * index's type. If the property is invalid or not a decorated property for
+     * the index, this does nothing.
+     * 
+     * @param p The property to remove
+     */
+    public void undecorate(Property p) {
+        Iterator<PropertyStore<?>> it = decoratedProperties.iterator();
+        while(it.hasNext()) {
+            if (it.next().property == p) {
+                it.remove();
+                break;
+            }
+        }
+    }
+    
+    /*
+     * An iterator implementation over the canonical componentDatas of the index.
+     */
+    private class ComponentIterator implements Iterator<Component<T>> {
+        private int index;
+        private boolean advanced;
+        
+        public ComponentIterator() {
+            index = 0;
+            advanced = false;
+        }
+        
+        @Override
+        public boolean hasNext() {
+            if (!advanced)
+                advance();
+            return index < componentInsert;
+        }
+
+        @Override
+        public Component<T> next() {
+            if (!hasNext())
+                throw new NoSuchElementException();
+            advanced = false;
+            return components[index];
+        }
+
+        @Override
+        public void remove() {
+            if (advanced || index == 0)
+                throw new IllegalStateException("Must call next() before remove()");
+            if (componentIndexToEntityIndex[index] == 0)
+                throw new IllegalStateException("ComponentData already removed");
+            removeComponent(componentIndexToEntityIndex[index]);
+        }
+        
+        private void advance() {
+            index++; // always advance at least 1
+            while(index < components.length && componentIndexToEntityIndex[index] == 0) {
+                index++;
+            }
+            advanced = true;
+        }
+    }
+
+    /*
+     * Type wrapping a key, property, and factory, as well as an auxiliary data
+     * store for compaction.
+     */
+    private static class PropertyStore<P extends Property> {
+        final String key;
+        final P property;
+        final PropertyFactory<P> creator;
+        IndexedDataStore swap; // may be null
+        
+        
+        public PropertyStore(PropertyFactory<P> creator, String key) {
+            this.creator = creator;
+            this.key = key;
+            property = creator.create();
+        }
+        
+        private void clone(int srcIndex, P dst, int dstIndex) {
+            creator.clone(property, srcIndex, dst, dstIndex);
+        }
+        
+        private void setValue(int index) {
+            creator.setDefaultValue(property, index);
+        }
+    }
+}

src/main/java/com/googlecode/entreri/Controller.java

     
     public void onComponentRemove(Component<?> c);
     
-    // FIXME: these interfaces are going to have to be changed, I would like to
-    // ensure that a controller is limited to a single system.  This might 
-    // require having Controller be an abstract class with some setSystem() logic, etc.
-    
-    // FIXME: part of this is because the amount of work to store system specific
-    // data into the CM's data store is super tedious if a plain field would work.
-    
-    // THe CM's data sharing works fine across controllers, although I will have to
-    // update the scene system to store things as collections instead of as certain
-    // components.
-    
-    // FIXME: adding profiling would be pretty cool too.
-    
-    // FIXME: REVIEW: Do we want to keep the phases for controllers? Is it valuable, useful?
+    // FIXME: TODO:
+    // Final solution: This is an interface, with a DefaultController or abstract, etc.
+    // that implements all of the methods as no-ops, except for init and destroy
+    // Part of the API is getEntitySystem(), which is used to validate by the controller-manager.
+    // rename addedToSystem, etc. to init(System) and destroy().
+    // I will keep the phases, since I can imagine a health controller that subtracts
+    // health during processing, and on post process removes everything that is dead
+    // TODO: I think I will add some Scheduler interface or type that is used to
+    // control the ordering and execution of controllers during a process request,
+    // theoretically this could do MTing, or dependency ordering. ATM it will do
+    // by-add ordering.
+
 }

src/main/java/com/googlecode/entreri/Entity.java

      * @throws NullPointerException if id is null
      */
     public <T extends ComponentData<T>> Component<T> get(TypeId<T> componentId, boolean ignoreEnable) {
-        ComponentIndex<T> ci = system.getIndex(componentId);
-        Component<T> c = ci.getComponent(ci.getComponentIndex(index));
+        ComponentRepository<T> ci = system.getIndex(componentId);
+        Component<T> c = ci.getComponent(ci.getComponentRepository(index));
         
         if (c == null || ignoreEnable || c.isEnabled())
             return c;
         if (data.owner.getEntitySystem() != getEntitySystem())
             throw new IllegalArgumentException("ComponentData was not created by expected EntitySystem");
         
-        ComponentIndex<T> ci = data.owner;
-        int componentIndex = ci.getComponentIndex(index);
+        ComponentRepository<T> ci = data.owner;
+        int componentIndex = ci.getComponentRepository(index);
         return data.setFast(componentIndex) && ci.isEnabled(componentIndex);
     }
 
      * @throws NullPointerException if componentId is null
      */
     public <T extends ComponentData<T>> Component<T> add(TypeId<T> componentId) {
-        ComponentIndex<T> ci = system.getIndex(componentId);
+        ComponentRepository<T> ci = system.getIndex(componentId);
         return ci.addComponent(index);
     }
 
     public <T extends ComponentData<T>> Component<T> add(Component<T> toClone) {
         if (toClone == null)
             throw new NullPointerException("ComponentData template, toClone, cannot be null");
-        ComponentIndex ci = system.getIndex(toClone.getTypeId());
+        ComponentRepository ci = system.getIndex(toClone.getTypeId());
         return ci.addComponent(index, toClone);
     }
 
      * @throws NullPointerException if componentId is null
      */
     public <T extends ComponentData<T>> boolean remove(TypeId<T> componentId) {
-        ComponentIndex<T> ci = system.getIndex(componentId);
+        ComponentRepository<T> ci = system.getIndex(componentId);
         return ci.removeComponent(index);
     }
 
      */
     private static class ComponentIterator implements Iterator<Component<?>> {
         private final int entityIndex;
-        private final Iterator<ComponentIndex<?>> indices;
+        private final Iterator<ComponentRepository<?>> indices;
         private final boolean ignoreEnable;
         
-        private ComponentIndex<?> currentIndex;
-        private ComponentIndex<?> nextIndex;
+        private ComponentRepository<?> currentIndex;
+        private ComponentRepository<?> nextIndex;
         
         public ComponentIterator(EntitySystem system, int entityIndex, boolean ignoreEnable) {
             this.entityIndex = entityIndex;
             
             currentIndex = nextIndex;
             nextIndex = null;
-            return currentIndex.getComponent(currentIndex.getComponentIndex(entityIndex));
+            return currentIndex.getComponent(currentIndex.getComponentRepository(entityIndex));
         }
 
         @Override
             while(indices.hasNext()) {
                 nextIndex = indices.next();
                 
-                int index = nextIndex.getComponentIndex(entityIndex);
+                int index = nextIndex.getComponentRepository(entityIndex);
                 if (index != 0 && (ignoreEnable || nextIndex.isEnabled(index)))
                     break;
                 else

src/main/java/com/googlecode/entreri/EntitySystem.java

  * @author Michael Ludwig
  */
 public final class EntitySystem {
-    private ComponentIndex<?>[] componentIndices;
+    private ComponentRepository<?>[] componentIndices;
     
     private Entity[] entities;
     
     public EntitySystem() {
         manager = new ControllerManager(this);
         entities = new Entity[1];
-        componentIndices = new ComponentIndex[0];
+        componentIndices = new ComponentRepository[0];
         
         entityIdSeq = 1; // start at 1, id 0 is reserved for index = 0 
         entityInsert = 1;
     }
     
     public <T extends ComponentData<T>> void setFactory(TypeId<T> id, ComponentDataFactory<T> factory) {
-        // FIXME: setting a factory will create the ComponentIndex, it cannot assign
+        // FIXME: setting a factory will create the ComponentRepository, it cannot assign
         // the factory if the index already exists
         // any other action that requires an index for a type will create the index
         // with the reflection default
 //        int minIndex = -1;
 //        int minSize = Integer.MAX_VALUE;
 //        
-//        ComponentIndex index;
-//        ComponentIndex[] rawIndices = new ComponentIndex[ids.length];
+//        ComponentRepository index;
+//        ComponentRepository[] rawIndices = new ComponentRepository[ids.length];
 //        
 //        for (int i = 0; i < ids.length; i++) {
 //            if (ids[i] == null)
      * @throws NullPointerException if type or factory are null
      */
     public <T extends ComponentData<T>, P extends Property> P decorate(TypeId<T> type, PropertyFactory<P> factory) {
-        ComponentIndex<?> index = getIndex(type);
+        ComponentRepository<?> index = getIndex(type);
         return index.decorate(factory);
     }
 
      * @throws NullPointerException if type is null
      */
     public <T extends ComponentData<T>> void undecorate(TypeId<T> type, Property p) {
-        ComponentIndex<?> index = getIndex(type);
+        ComponentRepository<?> index = getIndex(type);
         index.undecorate(p);
     }
 
     /**
-     * Return the ComponentIndex associated with the given type. Fails if the
+     * Return the ComponentRepository associated with the given type. Fails if the
      * type is not registered
      * 
      * @param <T> The ComponentData type
      * @param id The id for the component type
-     * @return The ComponentIndex for the type
+     * @return The ComponentRepository for the type
      */
     @SuppressWarnings("unchecked")
-    <T extends ComponentData<T>> ComponentIndex<T> getIndex(TypeId<T> id) {
+    <T extends ComponentData<T>> ComponentRepository<T> getIndex(TypeId<T> id) {
         int index = id.getId();
         if (index >= componentIndices.length) {
             // make sure it's the correct size
             componentIndices = Arrays.copyOf(componentIndices, index + 1);
         }
         
-        ComponentIndex<T> i = (ComponentIndex<T>) componentIndices[index];
+        ComponentRepository<T> i = (ComponentRepository<T>) componentIndices[index];
         if (i == null) {
             // if the index does not exist, then we need to use the default component data factory
-            i = new ComponentIndex<T>(this, id, createDefaultFactory(id));
+            i = new ComponentRepository<T>(this, id, createDefaultFactory(id));
             i.expandEntityIndex(entities.length);
             componentIndices[index] = i;
         }
         
         return i;
     }
-    
+
+    /*
+     * Create a new ComponentDataFactory for the given id, using the default
+     * annotation if available.
+     */
     @SuppressWarnings({ "rawtypes", "unchecked" })
     private <T extends ComponentData<T>> ComponentDataFactory<T> createDefaultFactory(TypeId<T> id) {
         DefaultFactory factoryAnnot = id.getType().getAnnotation(DefaultFactory.class);
             return new ReflectionComponentDataFactory<T>(id.getType());
         }
     }
-    
+
+    /*
+     * Look for a constructor that takes the given params and use it. Returns
+     * null if any exception is thrown.
+     */
     private <T> T attemptInstantiation(Class<T> type, Object... params) {
         Class<?>[] argTypes = new Class<?>[params.length];
         Constructor<T> constructor;
     /**
      * @return Return an iterator over the registered component indices
      */
-    Iterator<ComponentIndex<?>> indexIterator() {
-        return new ComponentIndexIterator();
+    Iterator<ComponentRepository<?>> indexIterator() {
+        return new ComponentRepositoryIterator();
     }
 
     /**
     
     @SuppressWarnings({ "rawtypes", "unchecked" })
     private <T extends ComponentData<T>> void addFromTemplate(int entityIndex, TypeId typeId, Component<T> c) {
-        ComponentIndex index = getIndex(typeId);
+        ComponentRepository index = getIndex(typeId);
         index.addComponent(entityIndex, c);
     }
     
 //    @SuppressWarnings({ "unchecked", "rawtypes" })
 //    private class BulkComponentIterator implements Iterator<IndexedComponentMap> {
-//        private final ComponentIndex[] indices;
+//        private final ComponentRepository[] indices;
 //        private final int minIndex;
 //        private final Iterator<ComponentData> minComponentIterator;
 //        private final ComponentData[] result;
 //        private boolean hasAdvanced;
 //        private boolean resultValid;
 //        
-//        public BulkComponentIterator(ComponentIndex[] indices, int minIndex) {
+//        public BulkComponentIterator(ComponentRepository[] indices, int minIndex) {
 //            this.indices = indices;
 //            this.minIndex = minIndex;
 //            
 //    
 //    @SuppressWarnings({ "unchecked", "rawtypes" })
 //    private class FastBulkComponentIterator implements Iterator<IndexedComponentMap> {
-//        private final ComponentIndex[] indices;
+//        private final ComponentRepository[] indices;
 //        private final int minIndex;
 //        private final Iterator<ComponentData> minComponentIterator;
 //        private final ComponentData[] result;
 //        private boolean hasAdvanced;
 //        private boolean resultValid;
 //        
-//        public FastBulkComponentIterator(ComponentIndex[] indices, int minIndex) {
+//        public FastBulkComponentIterator(ComponentRepository[] indices, int minIndex) {
 //            this.indices = indices;
 //            this.minIndex = minIndex;
 //            
 //                    if (i == minIndex)
 //                        continue;
 //