Blockchain: On chain vs off chain

In a previous post I spoke about how difficult/impossible it is to return an array in Solidity on an Ethereum flavour of blockchain. After having had a look at Hyperledger Sawtooth it seems this is a problem there too. I dug into the code used for Crypto Kitties which can be found here. An array is used to keep track of the Kitties Kitty[] kitties; but in Solidity if no access modifiers are defined for global variables then these variables have private access by default. I could not find any function that returns this array (which as mentioned in my previous post is not currently possible with Solidity).

But on the Crypto Kitties marketplace page multiple kitties are viewable as can be seen in the below screenshot.

This all indicates that they must be storing this data off chain. This is also the case with the Hyperledger Sawtooth examples I looked at. In those examples they use RethinkDB to store data to allow for faster and richer queries against data on the chain. This makes sense as a chain may have an array of data spread over multiple parts of the chain as data is added to the array throughout the lifecycle of the DApp. Hyperledger Fabric in fact has a database that is spun up with each node. This DB is responsible for recording state relevant to that node based on the node's privacy settings and channels they are subscribed to.

Based on all this it is clear that for DApps if you need to perform rich queries on chain data you will need to maintain a copy of it or portions of that data in an off chain DB. If you do not want to worry about managing this DB yourself and you are working on a permissioned chain DApp where you have the choice of chain technology to use then Hyperledger Fabric may be a good fit.