The Future of Ikaros:
A Roadmap
This text is an active document that will change as new ideas for the future version of Ikaros will be added. Last change 2011-01-20.
Introduction
Since it beginning in 2001, Ikaros has come along way towards realizing the initial goal of an infrastructure for system-level brain modeling. To date, over 30 people have contributed code to the project and many more have used it in various applications. After eight years, it has become clear how the system can be used and what users asks of it. In many cases, this has been what we initially thought, but in other cases, users have requested functions that were not at all anticipated.
Although, there was an initial focus on brain-like models it has become clear that Ikaros also has a place as general framework for robot control regardless of any relations to the brain. One aim for future versions is to add support for some functions that will make such use of the system easier.
There are also several additions that have been planned since the start but are not yet available. Below, I describe some of the future plans and considerations for Ikaros and gives some ideas about functions that will be added or changed.
Large Scale Computing
Ikaros was initially aimed at smaller computers or smaller clusters of computers. Since we have recently started to run Ikaros olso on supercomputers it has become apparent that better support for such systems is needed. Although it has always been the aim to be able to easily parallelize Ikaros simulations through specifications in the IKC-files that tells each module in which process to run, this is still not supported. We have instead used dedicated modules for inter-process communication. Although this works, it is unnecessarily complicated, and it is time to implement a more advanced support for multiprocessing. The current aim is to build upon MPI as a communication device and this will likely be the next major addition to Ikaros.
Larsge Scale - To Do
| Component | Author | Available | Included |
|---|---|---|---|
| Batch Element | CB | ✓ | 1.2 |
| Batch Combinatorics | CB | ||
| MPI | |||
| Parallel Batches | |||
| Collecting Statistics |
User Interface
There are two components of the user interface that are not yet available that will be prioritized in the near future. The first is interactivity and the second is the graphical editor for IKC-files.
Currently, the WebUI can only be used to passively view the operation of Ikaros. The only control of Ikaros that is available is to start or stop a process or to change view. In the future, we will add WebUI-objects that can handle interactivity. For example, it will be possible to add buttons or sliders or other input objects to a view. These objects will continuously change an input to a module to allow parameters to be set in real time or hardware devices such as robots to be controlled.
The second main addition will be a graphical editor for the IKC-files. This editor will also run in the browser and will make it possible to add modules and connections and to build hierarchies of modules interactively. With this addition, it will become possible to also use Ikaros as a teaching tool for people without programming skills.
WebUI - To Do
| Component | Author | Available | Included |
|---|---|---|---|
| HTML Shell | CB | ✓ | 1.2 |
| Separate Thread | |||
| Layout Engine | CB | ✓ | 1.2 |
| Inspector | |||
| Add Module | |||
| Add Connection | |||
| Saving | |||
| Controls | CB | partly | 1.2 |
Brain Models and Behavior Data
The major addition to Ikaros in the following years will be the addition of the actual brain models. There are currently a large set of modules that are not yet included in Ikaros. The inclusion of these modules in the system will make Ikaros useful as a demonstration tool for different brain models.
A related development will be public release of the behavior data that has been collected over the years. Initially, these data sets will be available as EXP-files. Later on, these files will be converted to a more general format that can handle a larger range of behaviors.
Database - To Do
| Component | Author | Updated | Included |
|---|---|---|---|
| EXP Parser | CB | ||
| Basic Database | CB | ||
| Extended Database | CB |
Data and Connections
General connections
One missing functionality that has often been requested is the ability to specify exactly how connections between modules should be made. Currently, Ikaros assumes that the order of several connections to the same input is not important. This is a reasonable assumption in the case of neural patterns, but does not work very well if the values are coded in some other way, for example as coordinates. A related problem is that it is not possible to only connect a part of an output to the input of another module. It would be useful if it was possible to specify ranges or individual components of an output to be used.
One future goal is to add two attributes to connections that can handle this. One possibility would be to use something similar to the syntax used for ranges of delays. For example, the following could be used to connected elements 2, 3, 4 of the output to elements 7, 8, 9 of the input:
<connection ... sourcerange="2:4" targetrange="7:9" />
However, this syntax would need to work also for higher dimensions. One possibility would be to use several ranges for different dimensions separated by, for example ";":
<connection ... sourcerange="2:4; 3:4" targetrange="7:9; 6:7" />
We could also list individual elements to be used:
<connection ... sourcerange="1,7,9" targetrange="4,1,2" />
The main problem would be to decide on how powerful this mechanisms would need to be. It is easy to come up with extensions that essentially would act as general for-loops, but perhaps this would lead too far. For example, the notation used in MATLAB for increments could be used where
1:2:10
would generate
1 3 5 7 9
It is also worth considering whether ranges should be specified within a string as in the examples above or as a number of attributes. Although the use of several attributes may more closely fit with the intentions of XML, it does feel a bit clumsy.
Higher dimensions
Another often requested functionality is matrices of higher dimensions than two. The very early versions of Ikaros could handle matrices of any dimension, but it was decided that ordinary two-dimensional matrices together with arrays would be sufficient. In retrospect, this may not have been the best decision. It has often been the case that sets of matrices are used and this would then need to be handled manually. It would be useful to include support for higher-dimensional matrices in Ikaros in the future.
However, this is just one instance of a larger problem. Sometimes it would be useful to have arrays of matrices of different sizes. For example to represent an image pyramid. It is not clear how this could be done in a general way though. Currently, modules that would need higher dimensional matrices use several inputs or outputs for the third dimension, but this solution is not always practical. It also has the disadvantage that the automatic scaling functions of the SetSizes() function cannot be used.
This is certainly an area were different options have to be evaluated before this functionality is added to Ikaros. It will also require a rewrite of some of the inner parts of the kernal that should be avoided as long as possible since they are likely to introduce problems that are hard to anticipate.
Coordinate systems
Many modules in Ikaros that use images use a coordinate system with values between 0 and 1 to refer to different locations in an image. This works very well in most case, but introduces problems when images are truncated as a result of filtering. For example, after smoothing an image with a 5x5 Gaussian kernel, its width and height will be 4 pixels smaller. This means that the coordinates are no longer accurate since a value of (0, 0) now refers to a location outside the image.
Although this is not necessarily something that Ikaros should take care of, it would be very useful if there were at least a standard for how to convert image coordinates between different modules to compensate for different operations. This is one area that will addressed in future versions of Ikaros.
Format for constants
To simplify the management of constant arrays and matrices it would be possible to use ";" indicate the end of a row. This would make it possible to infer the size of a matrix from its data and would make some parameters unnecessary.
Keeping Hierarchies
When an IKC-file contains hierarchies of groups of modules, they are currently flattened when the IKC-file is read. This makes it hard to debug errors in IKC-files with hierarchical groups. One future development of Ikaros will be to retain more of the heirarchical structure during execution. This is already the case with the XML-tree that is kept in memory, but it is not accessible in most cases.
This has been changed for version 1.2 where the hieararchical structure of a simulation can be accessed from the WebUI.
A New Binding Mechanism
Although it is easy to access parameters in an IKC-file through the GetValue-functions, but this mechanism has the disadvantage that it is not possible to change a parameter during execution. This is not a problem when Ikaros is run in a non-interactive way, but when Ikaros is run from the WebUI, it would be very useful if parameters could be changed interactively. To this end, it would be possible to use a binding mechanism that would bind a variable to a parameter which would case that variable to change if the parameter is changed. This could be accomplished with a call like:
Bind(&alpha, "alpha");
This would also allow a parameter to be connected to a graphical interface element such as a slider or a button. Since the type
This mechanism is so far implemented only for float values. It is trvial to extend it to other data types.
Bindings - To Do
| Type | Implemented | Included |
|---|---|---|
| float | ✓ | 1.2 |
| int | ✓ | 1.2 |
| bool | ✓ | 1.2 |
| list | ✓ | 1.2 |
IKC-Files
The transition to more extensive use of IKC-files is not yet complete. The goal is that things that are available in the IKC-files should not also be coded in the source code. For example, the names of inputs and outputs as well as oarameters should be taken directly from the IKC-file. This will make it possible to use the same creator functions for most modules. Some simpler forms of dependencies between input and output sizes could also be specified in the IKC-file which would make it unnecessary to write a SetSizes() function for simple cases.
The currebt version implelemts the follwing attributes ot set a fixed output size: size, size_x, size_y. To describe dependencies between input and output sizes the following attributes can be used: size_set, size_set_x, size_set_y, size_param, size_param_x, size_param_y.
Mixing the old and new style can be done by first calling Module::SetSizes() that will use the new attributes in a custom SetSizes() function that takes care of any additional sizes the old way. This is useful when there are many dependencies and only a few makes use of custom code.
Additional Modules
When version 1.0 was released, many modules were removed from the distribution since they were not sufficiently tested. Since the release, a large number of modules have been added and contributed from various authors but these modules are not yet part of the distrinution. One goal is to update all these modules to the new IKC-format and include them in future relases. The tables below list the available modules that are sufficiently general to be included in future distributions and their current status.
Environment Modules
| Module | Author | Updated | Included |
|---|---|---|---|
| ContinuousGridWorld | CB | ✓ | |
| MazeGenerator | CB | ✓ | 1.2 |
| BouncingBall | CB |
Learning Models
| Module | Author | Updated | Included |
|---|---|---|---|
| Balkenius98 | CB | ||
| BM_Context | CB,JM | ||
| Klopf88 | CB,JM | ||
| SuttonBarto | CB,JM | ✓ | |
| SchmajukDiCarlo92 | CB,JM | ||
| TD | CB,JM | ✓ | |
| ContextQ | CB,JM | ✓ | |
| ACQ | SW | ✓ | |
| ABF | CB | ✓ | |
| RLCore | SW | ✓ | |
| LinearNet | SW | ✓ | |
| Invert | SW | ✓ | |
| GreedySelect | SW | ✓ | |
| BolzmannSelect | SW | ✓ | |
| CRL | SW | ✓ | |
| CSFA | SW | ✓ | |
| KNN | AK | ✓ | Included |
| KNN_Pick | AK | ✓ | Included |
ANN
| Module | Author | Updated | Included |
|---|---|---|---|
| BackProb | MJ, SW | ||
| Perceptron | CB, AK | 1.2 | |
| RBF | MJ, SW | ||
| SOM | CB, MJ | ||
| Elman | MJ | ||
| CompLearning | MJ | ||
| HopfieldNetwork | MJ | ||
| ART1 | MJ | ||
| NeuralGas | MB | ||
| Trainer | CB |
Brain Models
| Module | Author | Updated | Included |
|---|---|---|---|
| M02_Amygdala | CB, JM | ✓ | |
| M02_Thalamus | CB, JM | ✓ | |
| M02_Hippocampus | CB, JM | ✓ | |
| M02_OFC | CB, JM | ✓ | |
| NeuralField | CB | ||
| HConSOM | CB | ✓ | |
| WorkingMemory | CB | ||
| SensoryCortex | CB | ||
| Automation | MA |
IO Modules
| Module | Author | Updated | Included |
|---|---|---|---|
| SCC32 | CB, MJ | ✓ | |
| Epuck | CB, BJ | ✓ | 1.1 |
| GyroController | CB | ||
| Vehicle | CB | ||
| IPCInput | CB | ||
| IPCOutput | CB | ||
| Axis205 | CB |
Vision Modules
| Module | Author | Updated | Included |
|---|---|---|---|
| ObjectRecognition | CB | ||
| ElasticTemplateMatching | CB | ||
| ActiveShapeMatcher | CB | ✓ | |
| StaufferGrimson | CB+ | ✓ | |
| OpticFlow | |||
| OpticTransform | CB | ✓ | |
| CircleDetector | CB | ||
| ContourDetector | CB | ||
| MonocularDepth | SK | ||
| Segment | JM | ||
| Tracking | JM | ||
| AttentionCompletion | SK | ||
| DiffuseDepth | SK | ||
| EdgeComplete | SK | ||
| FastTDetector | SK | ||
| HarrisCornerResponse | SK | ||
| LineEndDetector | SK | ||
| LineEndConnector | SK | ||
| LIneEndToContour | SK | ||
| OcclusionCmpleter | SK | ||
| TConnect | SK | ||
| TDetector | SK | ||
| FaceDetector | CB | ||
| EyeDetector | CB | ||
| EyeTracker | CB | ||
| ColorConstancy | CB | ||
| VisualFeatuesAndPositions | MA | ||
| VisualAttentionTest | MA | ||
| ColorConstancy | MA |
Control Modules
| Module | Author | Updated | Included |
|---|---|---|---|
| KalmanFiler | CB | ✓ | 1.2 |
| PIDController | CB | ✓ | 1.2 |
| PotentialField | BJ | ||
| AStarPathFinder | BJ | ||
| Gradient | BJ |
In addition to the modules above, there are many model specific modules that will eventually be updated and distributed through the Ikaros site. This includes many of the computational modules developed at LUCS.
Utility Function
Although Ikaros was build on the idea that there should be minimal required libraries, it has become clear that there are still enough string operations that need to be done that it would be useful to include a minimal string library. The same goes for XML DOM functions that were initially not included in the XML library but have been gradually added. It may be time to add a minimal set for more standardized DOM-functions to simplify the kerbel code.
| Type | Included |
|---|---|
| More standardized XML DOM functions | |
| JavaScript inspired string and array functions |