Get best of the week

Metastocle - decentralized data warehouse

image


What if you need to store a variety of data decentralized? Objects, arrays, dates, numbers, strings, yes anything. Is it necessary to come up with a powerful DBMS for this? Indeed, often we just need to store and receive data in a distributed, open manner, but as simple as possible and without any special claims.


In this article, I would like to reveal a little metastocle library , with which you can solve the above problem easily, but with some limitations.


A little background


About a year ago, there was a desire and need to create a music repository. This article details. From the very beginning it was clear that you need to write everything so that in the future you can do the same with other entities: books, videos, etc. It was decided to divide everything into layers that can be used independently.


Metastocle is one of the layers that allows you to store and receive many types of data (but not files), as opposed to the storacle layer , which implements the work with files.


, - , . metastocle. : , ...


, , :


  • — nosql , , ...
  • — , .
  • () — : , , ...

:


:


const Node = require('metastocle').Node;

(async () => {
  try {
    const node = new Node({
      port: 4000,
      hostname: 'localhost'
    });

    //  
    await node.addCollection('test', { limit:  10000, pk:  'id' });
    await node.init();
  }
  catch(err) {
    console.error(err.stack);
    process.exit(1);
  }
})();

:


const Client = require('metastocle').Client;

(async () => {
  try {
    const client = new Client({
      address: 'localhost:4000'
    });
    await client.init();

    //  
    const doc = await client.addDocument('test', { text: 'hi' });

    //   
    await client.updateDocuments('test', { text: 'bye' }, {
      filter: { id: doc.id }
    });

    //    
    await client.addDocument('test', { id: 2, text: 'new' });

    //   
    const results = await client.getDocuments('test', {
      filter: { id: 2 }
    });

    //   
    const doc2 = await client.getDocumentById('test', 2));

    //   
    for(let i = 10; i <= 20; i++) {
      await client.addDocument('test', { id: i, x: i });
    }

    //  ,   
    const results2 = await client.getDocuments('test', {
      filter: { id: { $gt: 15 } },
      sort: [['x', 'desc']],
      limit:  2,
      offset:  1,
      fields: ['id']
    });

    //  ,   id > 15
    await client.deleteDocuments('test', {
      filter: { id: { $gt: 15 } }
    });
  }
  catch(err) {
    console.error(err.stack);
    process.exit(1);
  }
})();

, , . , :


const node = new Node({
   port: 4000,
   hostname: 'localhost',
   collections: {
     test: { limit: 10000, pk: 'id' }
   }
 });

:


  • pk — . , . , , , uuid , .
  • limit —
  • queue — : ,   ,
  • limitationOrder — , . , , .
  • schema —
  • defaults — . 
  • hooks — . 
  • preferredDuplicates — .

(schema) :


{ 
  type: 'object',
  props: {
    count: 'number',
    title: 'string',
    description: { type: 'string' },
    priority: {
      type: 'number',
      value: val => val >= -1 && val <= 1
    },
    goods: {
      type: 'array',
      items: {
        type: 'object',
        props: {
          title: 'string',
          isAble: 'boolean'
        }
      }
    }
  }
}

utils.validateSchema() https://github.com/ortexx/spreadable/blob/master/src/utils.js


:


{ 
  defaults: {
    date: Date.now
    priority: 0
    'nested.prop': (key, doc) => Date.now() - doc.date
  },
  hooks: {
    priority: (val, key, doc, prevDoc) => prevDoc? prevDoc.priority + 1: val
  }
}

:


  • CRUD
  • Javascript, , .


javascript , . 


https://github.com/ortexx/metastocle/blob/master/dist/metastocle.client.js window.ClientMetastocle ...


Api


  • async Client.prototype.addDocument() —
  • async Client.prototype.getDocuments() — -
  • async Client.prototype.getDocumentsount() —
  • async Client.prototype.getDocumentByPk() —
  • async Client.prototype.updateDocuments() — -
  • async Client.prototype.deleteDocuments() — -

():


.filter — , :


{ 
  a: { $lt: 1 },
  $and: [
    { x: 1 },
    { y: { $gt: 2 } },
    { 
      $or: [
        { z: 1 },
        { "b.c": 2 }
      ] 
    }
  ]
}

.sort — , :


{ sort: [['x', 'asc'], ['y.z', 'desc']] }

.limit —


.offset —


.fields —


readme.



. : npm i -g metastocle --unsafe-perm=true --allow-root. , .


, metastocle -a getDocumentByPk -o test -p 1 -c ./config.js, 1 test. https://github.com/ortexx/metastocle/blob/master/bin/actions.js



  • , , , . , -, , -, , .
  • . , - , : , ... , , http , .

, .


, , :


  • , , , - , . 
  • - , , , . , 10000 - 100 , , , .

, , , , . , , .


:


More articles:


All Articles