Engine Workbench

Contains ideas, possible fixes and upgrades for the engine. Features should be distilled from this

• I just realized: if i add some dependencies to gameplay.dll, and hotload gameplay dll, can't i consider the dependencies as being hotloaded too? Is this an easy way to allow split dll hotloading?
• Support multiple gameplay dlls. (Also note the one core plugin loader idea)
• Note that the original idea was to have hierarchial hotloading, where not every subpart is hotloaded, in my quickfix above everything is relinked.
• Maybe support multiple gameplay dlls, to allow faster compiling, and put each testcase in a seperate dll.

• Engine Automatically remove all references when object is destroyed!
• Engine Use Expression framework of C# to update invariants automatically on change, detect what changes to listen for using this expression framework.
• Engine Maybe use postsharp aspect in conjuction with change model to automatically fix bidirecitonals?
• Engine Find something for bidirectional associations Engine Allow encapsulation of properties again in the modelobjects! (use direct field injection?)
• Engine Think about a 'looping simulator system', where methods are attached to object change events, and the simulator simulates until all change events are simulated!
• Engine A lot of bidirectional associations appear in my code: create something for this (cart holding item, item held by cart etc)
• Engine Think about extending to an event based system?
• Engine idea: there should be no relations between data fields in modelobjects (or the fields containing pure data should be indicated). Functionality should be added to allow properties that are forwards to other data. Automatic caching support for bidirectional (and maybe other) associations should be added!
• Engine Should these properties remain forced public as to force the coder to see these as possible access paths?
• ENGINEIdea: allow using interfaces as direct modelobjects. It is not necessary for them to to be implemented, automatic implementation (or simple public autoproperties) are probably good enough? Find a way to do this.
• Give each modelobject invariant functions, which are evaluated on reads. Writes are still dead, writing to modelobject properties does not invoke any other code except for change notification.
• Also: try this system by allowing empty setters, and fixed getters!
• Add safety check to see that ModelObjectChanged attribute was added to all IModelObjects !!!
• Create a List<> that fires changes to modelobjects
• Since simulators seem te become more like method binders to a specific order, maybe introduce a way to directly bind the methods instead of writing a new class every time, and provide new simulators which are Bundles of bindings for general usage.
• Idea: instead of making seperate code in tests for initialization and test code, simply block all TW.Data methods when not initializing??
• Automatically store all fields that contain modelobjects in a simulator in a “virtual” 'MySimulator.Data' modelobject.
• Idea: Dont allow global access to objects :) Isn't this actually a massive singleton global access to everything leak?? Maybe only allow global access to the parts? Think about this
• Idea: unreferenced objects could automatically be deleted (or garbage collected). All method an property calls could be changed to throwing an exception using postsharp when they are gc'ed.
• Think about adjusting the modelobject simulator system to a delayed event system:
• Now what actually is happening is that the simulators are observing changes on objects, but they process the events at a later rate. This could be extended to simulators who can listen to any event, but process their events in a delayed fashion, as to decouple the code causing the event and the code acting upon the event. This could allow for game-maker style events.

• Engine Allow DI in modelobjects!
• Engine Add constraint for multiple equivalent service implementations?
• Engine Apply DI on test classes!
• ENGINE Idea: create a IFactory<T> interfaces that represents a general thing that creates T's? Configure windsor to use the TW.Data.Get for dependency injection.

• Engine: allow resizing the engine, just keep aspect ratio of rendering and add some extra info below, so we can attach it to the side of the screen :)
• Engine: implement some form of autosaving, eg on switching tests.
• Switch to a real editor window? (windows does not handle multiple floating forms well) Pauzing simulation is handy
• Support premodifying of models (custom model format?) (it is currently not possible to scale imported models, since physx does not support dynamic scaling)
• Fix switch desktops for engine

Engine Automatic invariants: readonly fields are invariant? Think more closely: some fields are not supposed to change during runtime, but this is not always determined during compile time. Eg you could be using a class with a property which you set once and you know you don't want it to change during runtime, but that same class could be used somewhere entirely different where this is possible. Find a convention for this?

Engine Another hotloading idea: you can easily hotload at interface boundaries, by using a dynamic proxy that adapts to new interface implementations.

Engine Another style hotloading: make test setup deterministic, and use a ID that is counted using the object creation to identified identical objects.

Engine New style hotloading: per class! use a dynamic assembly, detect changes to a source file. Attempt to compile this sourcefile while referencing the old assemblies. Then only replace the changed class. Engine Disable simulators option for debugging at runtime Engine Show the contents of the line at which the error occured. This is a crazy usefull function when resolving nullreference exceptions due to missing windsor dependencies. Other idea is to simply use postsharp to say dependencies are not correctly set. Engine replace the singletons and static classes (TW,…) with dependency injection. Eg make public property public TW TW {get;set;} Engine Further look at the domain specific languages in visual studio. Can be used for all sorts of tasks to link between code and data outside of visual studio (eg assets to properties). Or can be used to compose entities? (eg put physical + itemstorage + model together visually) Engine Make test classes also modelobjects? Engine Modelviewer for the engine? Load model, exact visual as in engine, customizable environment, customizable lights. Engine Game loop improvements! http://lspiroengine.com/?p=378 http://gafferongames.com/game-physics/fix-your-timestep/ http://www.koonsolo.com/news/dewitters-gameloop/ Engine: Idea: can make implicit conversions from objects to objectparts and back?? Engine Add automatic utilities for objectParts Engine architecture: http://www.terathon.com/architecture.php http://forum.dronprogs.org/files_for_my_posts/books/GEA.pdf Engine Work out the part mechanism in more detail. Engine Add a groupsimulator? Engine Let the automated testing detect changes to objects (by creating wrappers for these objects perhaps?), then show a UI that allows checking all object interactions Engine Idea: use the expression framework for binding properties! (eg (o) ⇒ o.Value = repository.get(o).Value). It should be possible to create getters and setters from this using the Expression thing from c# Engine Idea: combining compile time code with run time created additional info: Add a GUID in code to a object, for which properties are set in code. At runtime this object can be changed, but its info is stored with the guid, so that next time the program starts these code and runtime elements can be recombined! ENGINE Idea: it is best NOT to use the simulator framework when implementing gameplay. However, you could implement the services the gameplay components use using the simulator framework (eg wrap Multiple services into simulators). In this scheme the service implementations used are facades to the simulator framework (by for example interfacing with the ModelObject data layer for communicating with the simulators,which is what you would do when directly using simulator framework. This allows for reusable gameplay components and at the same time you can use higher level service implementations! Other way to do this: simply make higher level service implementations :p ENGINE Idea: provide a service that allows 'simulating'. This service (used for DI) provides a method to attach a 'simulate or update(elapsed)' method, which is then called by this service when simulation is required. This way not all components that use time based simulation have a public Update(elapsed) method, which i think is not so good. Something similar could be done for Render()? Gigantic flaw in simulator system: constructors (for modelobjects!) and also using new simulator is a problem in the tests. The original idea was that using new was handy, and could be allowed by using postsharp to inject the constructor. This allows that the new call can do some extra behaviour, which would be the goal when using a factory. The problem now with this approach is that the non-engine code now will get very complicated if it wants to have some kind of 'factory' for modelobjects, using new. It will just use normal factories, which makes this complex construction useless. Try to refrain all use of new and use InversionOfControl + factories. Warnings toevoegen bij vergeten simulators DI Assign TW.Data.Get to simulators automatically using dependency injection?, sounds very handy (eg add field public CameraInfo camInfo, automatically loads tw.data.get<CameraInfo>()) Modifying code layout in with visual studio extensions: http://msdn.microsoft.com/en-us/library/dd885121.aspx When using dynamic modelobjects, earn in runtime which ObjectParts(Components) a type of ModelObject has and use this information during design time for intellisense! Add some runtime learning things, like learning how an IModelObject looks by detecting that it becomes physical and then using that mesh to visualize it. Should reaaaally add some form of automated testing, since now operational code sections are a big questionmark BUG ModelObject with parameter in constructor crashes the engine :) Adjust system to also support features in this: http://www.gamedev.net/page/resources/_/technical/game-programming/understanding-component-entity-systems-r3013. This is the component extension system previously mentioned, but notice that behaviour depends on collections of components, which is actually what simulators should be doing by definition. Think about this for the component framework: https://github.com/ekonbenefits/impromptu-interface. (bind dynamic object to component interface) Options for component framework: Create permutation interfaces of all Icomponent (eg IPositionVelocityGoblinCommandHolder) Then TW.Data.All<Position,Velocity,GoblinCommandHolder> returns of type IPositionVelocityGoblinCommandHolder, so that we know which elements we have. The All method could only accept max 4 components. More components can be used by combining them into interfaces which have meaning like IPlayer which has Position, velocity, etc? This greatly reduces the number of permutations. Other option might be to use dynamic objects + visual studio extension for intellisense http://msdn.microsoft.com/en-US/library/microsoft.visualstudio.language.intellisense.icompletionsourceprovider%28v=vs.100%29.aspx http://msdn.microsoft.com/en-US/library/microsoft.visualstudio.language.intellisense.icompletionsource%28v=vs.100%29.aspx http://msdn.microsoft.com/en-US/library/dd885244%28v=vs.100%29.aspx The whole simulator construction starts to seem like a 'custom gameloop' for each test, where the sequence and ordering of the gameloop is determined by the test, not by the gameplay itself. Changing another property than the world matrix for an entity every frame will cause the simulated info from physx to get overwritten for dynamic objects, since the engine currently can't see which property has changed! Add warning for non idempotent simulator: nothing should change on a second hotload if code wasn't changed. Not that does only counts for non time based simulators, maybe make exceptions here. Think if some more checks and warnings for using the correct programming paradigm in the engine. (eg no code inproperties) Why did i make propertys the core construct for modelobjects, if i am not allowed to put anything in the getters and setters? Doesn't this describe fields?? With the current design it is not possible to rename/double bind properties. (for example binding a position to a worldmatrix somewhere else). This is due to the fact that direct propagation of data is not allowed, all propagation has to occur clocked. This might need some exception( for example double binding properties for renaming, etc). Maybe this might actually be a good thing? Detect changed models in engine Add support for storing an 'inital' test engine data and a 'in progress' test engine data. Support building tests in the editor? This is part of making the editor feel more like a sandbox = living persistent world! http://obviam.net/index.php/the-android-game-loop/ Think about debug visualizations using attachment. Serializer still causes crashes when inappropriately used HOTLOADING By using the concept of 'reloading everything' and 'linking and unlinking' TW.Data, it is now actually possible to simply hotload all code. This is of course the same as restarting, but by creating linking layers between dlls it would be possible to only reload the changed dlls, allowing for increased productivity. Maybe it is also possible to create a [Hotloadable] postsharp aspect that makes a class hotloadable. RULE definitely add 'No manipulation of TW.Data in constructor of modelobjects. These constructors can be called anytime by the engine and should therefore not affect any other objects than themselves' RULE only allow serializable properties on modelobject's! RULE Detect forgetting of [ModelObjectChanged] attribute. Old way of persistence: DataStore: contains all to serialize object PersistanceScope in Datawrapper adds new objects to datastore PersistanceScopeAttribute enters persistancescope Figure out a garbage collection mechanism for dataobjects! Maybe put dataobjects in containers, based on where they are created (simulators, engine?, …) RULE Creating modelobjects in modelobject constructors is not allowed since it will cause the objects to become unreferenced. Idea: automatically create simulators from modelobject public methods which have an attribute. Ordering could also be specified in this attribute Really add automated testing, it should not even test something, but at least see that every test runs without exceptions. Note that the engine catches all exceptions, so these should be handled somewhere somehow. Maybe add a general service to log exceptions to that are handled, but really are exceptions and should be logged or sent to the developer. HOTLOADING Figure out a way to make simulators data objects?? This could allow for crazy hotloading maybe? Add support for writing test methods with arguments, which accept multiple paramters (instead of making sepereate test methods) HOTLOADING Problem: loading saves with singletons causes old singleton objects to remain lingering, thus effectively creating a TWO singleton instances. This should be resolved by NOT removing singletons when loading the TW.Data from file, but replacing the data in these singletons by the new data! This requires tracking of singletons! HOTLOADING Another option would be to add guids to all objects ,and to update objects if they already existed, but this seems maybe to complex? RULE add coding rule + find automatic constraint: simulators cannot store modelobjects in fields, because on a hotload, this data is lost and will cause trouble. =⇒ solution: add a Data class to store this RULE add coding rule: check if simulators are invariant!! This can be done by setting the elapsed to 0 and running the constructors and simulate methods multiple times, and see of changes occur. HOTLOAD TODO: Provide reload functionality when loading the plugin from gameplay dll fails! HOTLOAD TODO: add a mechanism to the TW.Data that allows storing data for modelobjects, for which the class is not loaded. This is some sort of variant type which can hold the fields, but is transparent for the rest of the system. This means that serializing this variant objects, and then later deserializing them again, this time with the class present, keeps everything operational. The goal of this variant type is that no data is lost when classes are missing! TODO: put spectator camera data also in the TW.Data! Check the persistancescope attribute, it might be unused, broken, or whatever :) Note that the current goal is to persist everything, simulators should be able to fix any discrepancies due to persistance. Add some way to provide ordering rules for simulators. Maybe even do automatic ordering in the engine. TODO: test this ⇒ In the VSDebugAttacher, check if attached already right before attaching! The current modelobject part's mechanism is a form of multiple inheritance through composition/adaption dd a gamemaker 'sprite' concept (entity template or smth). Support a simple in-engine editor to put markers and things like that. Support 3D text rendering (billboard) RULE A test should not change TW.Data,only the setup part should! DATA Maybe add a mechanism to prevent changes to TW.Data allow to create groups of simulators, and allow these groups to run multiple times each frame will there are still changes! Think about going back to the 'interfaces are created when a responsibility needs them, not when they seem to be needed by a number of responsibilities'. Also think of making one set of interfaces between gameplay logic and other logic of the game. Gameplay logic consists only of AI, where AI is defined as added intelligency by the system. This includes: a tree is intelligent in that it is able to make resources. Note that this makes an extra decoupling layer between the simulator system, and the gameplay, whereas now the simulator system complete encompasses the gameplay. It might be possible to find a way to automate this coupling. Maybe the genious plan might be to make data objects interfaces instead of classes? HOTLOADING Another thought about hotloading: Services are hotloadeable, since they can be identified by the fact that they implement a certain service!? Provide intellisense support for assets by autogenerating classes using T4 templates?! Note that missing assets will prevent compiling if this is used. Maybe take a step in between by creating a 'sprite' layer, for which files are small and can be included in the source. HOTLOADING The main thing that is required for hotloading is, a way to identified instances after the hotload and link them to instances before the hotload! If we know this, we can start mapping old data to the new structure!.

• Let wizards engine compile, disable edit and continue in vs, this could potentially allow continuous debugging!!!

* Disable simulation when engine is not active window (with optional button) * Maybe automatically restart debug

Maybe even add global hotkeys to the engine for when it is not active? Fix nunit slorping up errors When throwing exceptions in getters, the exception is displayed using the IErrorLogger. URGENT NUnitTestRunner add use of the ExceptionLogger!!!! (DI move to core) Bug: Fixed WorldMatrix = matrix.identity missing from physical

* Let hotloading work on fields instead of properties? Or should these properties remain forced public as to force the coder to see these as possible access paths? * Create a 'kill-thread' which monitors the thread that runs gameplay simulation (maybe even the rendering thread), and aborts it when it fails to send regular heartbeats. * Fix resharper settings. + syncing of templates