The characteristics and consequent limitations of distributed data design

Abstract

The ability to scale up applications is a definite requirement when there is an increase in operational demands. Scale-up is aided by providing computational resources and data replication. Nonetheless, scale-up fails, for example when there is data contention. Whilst partitioning and replicating data increases the reliability and performance, replication also add higher demands on maintaining consistency across data replicas. Data contention, in this case, is when having transactions overlap on common data and same data is over many replicas. Concurrency correctness is controlled by ordering the transactions, but it still cannot address transaction rates - i.e. delays. Many database management systems (DBMS) are available. For this project, MongoDB was used due to its reputed reliability and good performance through shards and replicas and also it adopts a JSON data model; JSON is very popular on web apps. The thread of this project was to create a tool which helps app developers with JSON manipulation and also to provide an easy way to perform replication and sharding set-up procedures. This tool targets users with minimal expertise in MongoDB's physical and networking design and hence, the user does not require to know anything other than to have a little background knowledge about MongoDB. The implementation of this project involved in creating a tool with a graphical user interface similar to a wizard's design, which generates shards and replicas. The tool gives the user two options, they can either import a JSON file to a MongoDB database, or choose to view a list of existing databases on which they can perform sharding and replication procedures. Various features were added to the tool, always ensuring that the design is kept as simple and as easy on the eye as possible.