X-Git-Url: http://www2.svjatoslav.eu/gitweb/?a=blobdiff_plain;f=doc%2Findex.html;h=b6bc9d1621830360b1e65fd0f6bfa84a55d6e729;hb=c20ef78b553cad795177f4415a20ba8f581426b9;hp=d0f979d5b047cfde09b816345d0080da6f77137e;hpb=32e305d7f189ddd3a6e62a2abf29065187cc75b2;p=sixth-data.git diff --git a/doc/index.html b/doc/index.html index d0f979d..b6bc9d1 100644 --- a/doc/index.html +++ b/doc/index.html @@ -1,418 +1,603 @@ - - + + + -Sixth - system for data storage, computation, exploration and interaction - - - - - - - -" + + + +Sixth Data - Data storage and computing engine + + - + + + + + -
-

Sixth - system for data storage, computation, exploration and interaction

-
-
+ +
+

1.1 Source code

+
+
+
-
-

2 Inspiration

+
+

2 Vision / goal

+

+Provide hackable, versioned, optimized, distributed, geometrical, +arbitrary dimensional (hypercube based) data storage and computation +engine (as inspired by the brain) for general purpose visual computing +environment called Sixth. +

+ +

+Because Lisp is hackable self defined programmable programming +language it would be used to provide imperative programming support. +

+
+
+
+

3 Inspiration

+
    -
  • Relational databases: -
      -
    • Transactional. -
      • -
    • Indexable / Quickly searchable. -
      • +
    • see also: OLAP cube.
    -
    • +
+
+

3.1 Brain

+
+ + +
  • Such properties allow parallel Geometrical computation and +beautifully fits CM-1 Connection Machine architecture (for extra +hardware accelerated solution).
    • +
    -
    -

    2.1 Brain

    -
    -
      -
    • Appears to have more than 3D dimensional design. Food for -thought…) + +
      +

      3.2 CM-1 Connection Machine

      +
      +

      +https://en.wikipedia.org/wiki/Connection_Machine +

      + +

      +Massively parallel (thousands of CPUs) connected via +machine's internal 12-dimensional hypercube network allows to +efficiently simulate arbitrary dimensional hypercube and network +topology between computational units. So that when we are +solving/simulating for example 5 dimensional problem, we can arrange +computational units into virtual 5D network. See: +http://www.mission-base.com/tamiko/theory/cm_txts/di-ch2.html +

      + +

      +we can pre-distribute data across computation units and perform +parallel geometrical computation. +

      +
      +
      +
    + +
    +

    4 Reasons for hypercube as a so called first class citizen

    +
    - +
    +
    +
    +

    5 Geometrical computation idea

    +
    +
    +
    +

    5.1 Distributed computation and data storage

    +
    +

    +Lots of problems can be translated to geometry (use any shapes and as +many dimensions as you need). Solution(s) to such problems could be +then found via geometrical search/comparison/lookup results. As a +bonus, such geometrical data storage AND computation can be +naturally made in parallel and distributed. +

    -
  • From there come following ideas: -
      -
    • Maybe every problem can be translated to geometry (use any shapes -and as many dimensions as you need). Solution(s) to such problems -would then appear as relatively simple search/comparison/lookup -results. As a bonus, such geometrical *data storage* AND -*computation* can be naturally made in *parallel* and -*distributed*. That's what neurons in the brain appear to be doing -! :) . Learning means building/updating the model (the hard +

      +Learning means building/updating/re-balancing the model (the hard part). Question answering is making (relatively simple) lookups (geometrical queries) against the model. -

      • +

      +
    • +
    +
    +

    5.2 Mapping hypercube to object-oriented model and relational database

    +
    +

    +Object oriented programming is inspired by the way human mind +operates. It allows programmer to express ideas to computer in a more +human-like terms. +

    + +

    +It is actually also possible to map object model and relational +database to geometrical hyperspace: +

    -
  • Mapping of hyperspace to traditional object-oriented programming -model:
      -
    • Object is a point in space (universe). Each object member -variable translates to its own dimension. That is: if class -declares 4 variables for an object, then corresponding object +
    • Object or database table row is a point in hypercube arbitrary +dimensional space. Each object member variable or database table +column can be mapped to its own dimension in hypercube. That is: if +class declares 4 variables for an object, then corresponding object can be stored as a single point inside 4 dimensional -space. Variable values translate to point coordinates in -space. That is: Integer, floating point number and even boolean -and string can be translated to linear value that can be used as -a coordinate along particular dimension. -
      • +hypercube. Variable values translate to point coordinates in that +hypercube. That is: numbers and string can be translated to linear +value that can be used as a coordinate along particular dimension. -
    • Each class declares its own space (universe). All class -instances (objects) are points inside that particular -universe. References between objects of different types are -hyperlinks (portals) between different universes. -
      • +
    • Each object class or database table declares its own hypercube that +contain instances (objects) of that class or rows of a table.
    -
    • +
    +
    + +
    +

    5.3 Mapping entity relations in hypercube

    +
    +

    +Consider we want to create database of: +

    +
      +
    • Books.
      • +
    • Authors.
      • +
    • Effort: Amount of time contributed by every author to every book +that he/she wrote.
    - + +

    +Information above can be represented as 3D cube where dimensions are: +

    +
      +
    • X: Book
      • +
    • Y: Author
      • +
    • Z: Effort
    + +

    +Points in that cube would nicely capture many to many relations +between authors and the books. +

    - -
    -

    3 Current status

    -
    +
    +

    6 Current status

    +
      -
    • More or less defined Vision / goal. -
      • +
    • More or less defined Vision / goal.
      • -
    • Collected some ideas. -
      • +
    • Collected some inspiring ideas.
    • Implemented very simple persistent key-value map.
      • Long term goal is to use it as a backing storage engine and -implement more advanced features on top of this. -
        • -
      -
      • +implement more advanced features on top of this via layered +architecture. +
    -
    -

    4 See also

    -
    +
    +

    7 See also

    +

    Interesting or competing projects with good ideas:

      -
    • GRAKN.AI: database in the form of a knowledge graph that uses -machine reasoning to simplify data processing challenges for AI -applications. +
    • flexible user interface building for interacting with different data
        -
      • https://grakn.ai/ -
        • +
      • Glamorous Toolkit +
          +
        • Moldable development environment. It is a live notebook. It is a +flexible search interface. It is a fancy code editor. It is a +software analysis platform. It is a data visualization engine. All +in one.
          • +
        • +
    - +
    -
  • Gemstone/S based on Smalltalk. +
    • +
  • Array Databases: Concepts, Standards, Implementations
    • -
  • Magma distributed database in Smalltalk. +
  • TileDB -
    • +
  • Analyze and share complex multi-dimensional data at scale
    • + + +
  • CM-1 Connection Machine
    • + +
  • Lisp-Stat: An environment for Statistical Computing
    • -
    -
    -
    -

    Author: Svjatoslav Agejenko

    -

    Created: 2017-07-19 Wed 21:46

    -

    Emacs 25.1.1 (Org-mode 8.2.10)

    -
    +
    +
    +
    +

    Author: Svjatoslav Agejenko

    +

    Created: 2021-04-09 Fri 16:55

    +

    Validate

    +