Update – What is happening

We no longer update this blog. To stay up to date with Komodo follow us here:

P.S. We are working on a new website for Komodo Platform!

Decentralized Key/Value Storage

Background Info

What is a key/value store database?

Blockstack Server: A Key-Value Store on Bitcoin


just made KV assetchain, will be added to list of coins
it will have the smart function of Key Value storage
took 15 minutes from concept to notaries funded with KV coin so they can dPoW

but it has no smart functionality yet, that will be added as I can, in order to solve some DEX and PAX issues, I need a KV storage, so I figured might as well make a decentralized KV to help out.

KV storage is a low level way a lot of things that need a database do things
KV chain will be a decentralized KV storage that can be used by anything that uses the KV API

decentralized data storage is relatively important as it allows for long term state to be securely created and modified, it will mimic an append only database, so changing data will be supported.

There are 1 million KV coins and its value will reflect the value of decentralized KV storage. Of course, KV is dPoW’ed and supports zcash shielded transactions. Record size will be limited to 8192 bytes, but multiple records can be combined to create larger sized storage. txfee is standard bitcoin txfee model, and if the capacity becomes an issue, a fee market will develop.

In order to reward the long time SuperNET investors, I will be sharedropping 20% of KV coin to SuperNET coin holders. We will need to create a migration path from the SuperNET NXT asset to the SUPERNET assetchain first,but that is already in the works for the migration of the various NXT assets. Once that is done, then we can do a KV sharedrop. You must hold the SUPERNET coin (not NXT asset) to get the sharedrop as there is no way to directly sharedrop a KMD assetchain coin in NXT.

If you are interested to be a part of KV coin team, please contact me, I certainly wont have time to do much with KV coin (other than technical) and I think it has a relatively large possibility for usage due to the popularity of KV storage. KV coin bounties and even some KMD will be made available

KV coin is an example of how I see the komodo ecosystem developing. As coins such as KV get established and succeed, KMD will gain value as the underlying coin.


The KV functionality didnt take long to add to komodod. It works for all the blockchains, komodod and all its assetchains. This allows each chain to have its own KV storage.

Initially, it does no protections for keys, so anybody can update any key at any time. Still thinking about the best way to give update rights to the creator and not to others. I am leaning toward having a signature in the value portion. That way we can use an external signature system to manage this. Using utxo sigs just seems a bit too messy.

The API is as simple as can be:

kvupdate key value
kvsearch key

For now, everything is a string, didnt want to complicate things to deal with binary key/values. It does waste about 12% of space, I can always add encoding modes later. The following is a simple test sequence. I first assign the value of “bar” to the key “foo”. Then I overwrite it with the value of “spam”

To make the behavior identical on all nodes, until the block with the KV entry is processed, it wont take effect.

See code block


I added support of a flags parameter to kvupdate.

The least significant bit is a write protect bit, so if it is set, it cant be updated until it expires.

The next bit is reserved, will become a binary data flag for signed data

The rest is currently use as a duration multiplier. The normal fee is multiplied by the the duration multiplier. The default is 1 day (1440 blocks)

Using the write protect bit, it is a bit of a first come first served free-for all, but with the duration all things will expire when the payment is used up.

I think with a signed binary option, we can get all that we need as the signed entries wont be able to be changed without a valid signature.


Bitcointalk Topic Entries: 1, 2, 3

An Atomic Swap through a Basilisk <-> Basilisk Mode

I changed the basilisk lower level code to use the new DEX* API as it is much much faster and supports [“KMD”, “BTC”, “USD”, “EUR”, “JPY”, “GBP”, “AUD”, “CAD”, “CHF”, “NZD”, “CNY”, “RUB”, “MXN”, “BRL”, “INR”, “HKD”, “TRY”, “ZAR”, “PLN”, “NOK”, “SEK”, “DKK”, “CZK”, “HUF”, “ILS”, “KRW”, “MYR”, “PHP”, “RON”, “SGD”, “THB”, “BGN”, “IDR”, “HRK”, “REVS”, “SUPERNET”, “DEX”, “PANGEA”, “JUMBLR”, “BET”, “CRYPTO”, “HODL”, “SHARK”, “BOTS”, “MGW”, “MVP”]

Since that includes both BTC and KMD, it means a node can quickly get up and running. Of course, if possible, it is always better to run the fullnode locally, but somethings that isnt possible or practical (or convenient).

As I wrote before, there are 3 different modes a coin can be in as far as iguana is concerned: native coind, iguana full mode, and basilisk mode. The latter is now using the DEX* API and will only work with the above coins.

To do a DEX swap, that is 3×3 combinations and the hardest is a basilisk <-> basilisk as all the info needed to construct the rawtx must come from a remote node. It does all the tx signing locally though and it also queries multiple nodes for the most sensitive information (value of utxo) and with all signing done locally, by the time it is transmitted to the network it is a fully signed tx

The swap: MVP <-> USD

Setup LP node:
cd tests
./paxfiats 1
cd ..
coins/basilisk/mvp
curl –url “http://127.0.0.1:7778” –data “{\”agent\”:\”bitcoinrpc\”,\”method\”:\”walletpassphrase\”,\”password\”:\”putpasswordhere\”,\”timeout\”:86444}”
curl –url “http://127.0.0.1:7778” –data “{\”agent\”:\”tradebot\”,\”method\”:\”amlp\”}”
curl –url “http://127.0.0.1:7778” –data “{\”agent\”:\”tradebot\”,\”method\”:\”liquidity\”,\”targetcoin\”:\”MVP\”,\”vals\”:{\”rel\”:\”USD\”,\”bid\”:0.09,\”ask\”:0.11,\”maxvol\”:100}}”

The LP node needs to load USD and MVP chain, login to a specific wallet, notify iguana that is it an LP node with “amlp” API and then submit liquidity commands. In this case a bid 0.09/ask 0.11 for MVP/USD

Setup client node:
coins/basilisk/mvp
cd tests
./paxfiats 1;
cd ..
./walletpassphrase

The client side is simpler, just need to load the 2 coins and login to a wallet.

Additionally, both sides needs to make sure it has enough utxo to complete the atomic swap. there are fees to pay, an extra deposit for the “bob” side, so it is recommended to have a common transaction size utxo and to make a set of them:

fiat/usd sendtoaddress <youraddress> 1
fiat/usd sendtoaddress <youraddress> 1
fiat/usd sendtoaddress <youraddress> 1
fiat/usd sendtoaddress <youraddress> 1
fiat/usd sendtoaddress <youraddress> 1

Now we are ready to start the DEX swap with the client side doing:
curl –url “http://127.0.0.1:7778” –data “{\”agent\”:\”InstantDEX\”,\”method\”:\”request\”,\”vals\”:{\”source\”:\”MVP\”,\”amount\”:0.1,\”dest\”:\”USD\”,\”minprice\”:0.09}}”

This command is sent to the DEX network and the LPnode sees that it fits its requirements and it responds with an offer. The client node waits for 5 seconds and accepts the best offer that has at least its minimum and the atomic swap protocol is started:

sendrawtransaction myfee.(4bf13c090d388d246b6425bc15faaa8be93d392d563310ca40fb1e03100dec25)
sendrawtransaction myfee.(2e83dd0efd58fd1b0c485f1e3d57747c8f1ca569413e737a2536e9343abb5965)

sendrawtransaction bobdeposit.(85972780232abf5d8e146f2fc868c19b35d2c207da8289a86540c13fe29aaa54)
sendrawtransaction alicepayment.(52c2cfdabb159e0dd88351008b4b49300a876c563a03eed3094d4372a24610fa)
sendrawtransaction bobpayment.(28bfd8d8be5ba3dda86e2db81438f04da026ce9ab8545c1eaf570134130427f1)
sendrawtransaction alicespend.(c4975c90929691b28181eb2439c40f6207355b0b1e790e4fd0cd6ed92069821a)
sendrawtransaction bobspend.(18441848573d7d145754078a0d904d91c1f45ec24188655d2a5ffb88567c7bbc)
sendrawtransaction alicereclaim.(e535d78c4cbf6970a71933db08f3c1240594792f61457baacff4d889ad7bdf37)
sendrawtransaction bobrefund.(8c6b1524caa1f58f5a644ead14cb96290ea274215621cba6da7935e273d08e0e)

Still a few issues, in this case only the MVP transactions made it to the blockchain, need to see why the USD ones didnt. But it is almost ready for GUIfication. Once the basic atomic swap is automatically matched and completed, then it will still need a lot of error checking and especially recovery from the error states. I estimate 95%+ of DEX transactions that are started will complete properly.

At this point I dont think we will have a DEX GUI completed when mainnet becomes official, but as you can see the low level command line level is functional even with basilisk <-> basilisk


Fixed the bug on failed remote sendrawtransaction. Here is an automatically matched and completed DEX:

Fees:
sendrawtransaction bob myfee.(b63921288ae1e95a86784d64f3b20d0579479fa64bcaf986e962fa9f45ec0ceb)

sendrawtransaction alice myfee.(b0fc0614ab06cb1e919a62ec0157d990820fef33f7e82ee5139e2829ec097f06)

Payments:
sendrawtransaction bobdeposit.(e871c601d81c3d7a0fbe5be75299941378a106249bc58498a6d0dd1e510b301d)

sendrawtransaction alicepayment.(4b7bf772e4cca57de257121f1aa3165898e3d0006b31f9fd57db60149cd0c1f0)

sendrawtransaction bobpayment.(97bd46d2bbd802722aac11da2426463be2f753a4298f361e1ac28a88d44b44f1)

Spends:
sendrawtransaction alicespend.(ea61c789f75d7bbe93faf0e5690d2a5ccafda4696ab754f2b7af7a6367146b18)

sendrawtransaction bobspend.(395ede337f84a99f5ba33d152263d6a5f5314d1cd823fecb956f324ac25083a6)

Refund of deposit

sendrawtransaction bobrefund.(00478df7ff5f9571014976a5968d91f293e27486e7ebba5b815adc354d9337f0)

[See the original bitcointalk post for the complete code blocks]

Failure of reclaim (this is good as alice spent her half):
sendrawtransaction alicereclaim error.({“result”:null,”error”:{“code”:-26,”message”:”18: bad-txns-inputs-spent”},”id”:”jl777″}

automatched DEX trade completed properly and all required transactions got confirmed in their respective blockchains. At no time was either party in a situation where they are exposed to large financial loss. There are some edge cases that a statistical process uses to make it so that anybody that tries to cheat will end up losing money and the other party will benefit from these losses.

All the other deployed DEX implementations I know about use an intermediate token and only partially decentralizes the process. Trust is required in the issuer of the token and while this is a step up from a central exchange, as the token issuer is not involved in the exchange process directly, it is a single point of failure. MGW distributes this single point to multiple, but still having an intermediate token is one token too many.

When people want to trade A <-> B, there is no room for C.

iguana’s native DEX allows this direct A <-> B trading. One structural issue with a native DEX is what to do with partial fills. In a central exchange, this is not any big problem, but with a native DEX, each swap between two accounts takes the same amount of transactions, regardless of the amount swapped. And due to the increased transactions needed the txfees can easily dominate. That is no good for anybody but the miners, but it also bloats the blockchain with small transactions.

What I came up with is the LP (liquidity provider) node auction. It is a fill or kill auction that the end user node starts and the LP nodes need to fill the entire order to make a valid offer. By this method, partial fills are sidestepped. Basically all the swap will happen or none of it.

The end user realtime auction acts a bit like a market order, but you can specify the minimum price you will accept so there is no danger of getting a large amount of slippage. And the end user node will find out in seconds if there is any LP node that will fill the order. Even if it takes a while (bitcoin confirmation time if BTC is involved) to finalize the transaction, having another party initiate the other side is a very good indication that the trade will complete.

To put long term bid/ask prices, you do need to run an LP node. However, via the basilisk mode you can be an LP node without having any coin running locally. If you will be an active market maker it is strongly advised that you run all coins locally, but if you just want to place occasional bid/ask orders you can invoke the LP node mode for this without having BTC locally. You will have to keep your node online for the duration that your bid/ask is active, or in other words if your node goes offline, so does your bid/ask. The reason is that there is no central server and your node is responding to the realtime auction requests.

OK, so now you ask about privacy. A good question, but there is a layer in between all traders. While logically, a trade is between end user node <-> LP node, under the hood, it is actually like this:

end user node <-> DEX network <-> LP node

The DEX network is comprised of the notary nodes that run an interleaved PUB/SUB network along with a REQ/REP service. This allows for a single request to be submitted to the DEX network and have it propagate not only to all other notary nodes, but to all subscribers to the DEX network (active end user and LP nodes). The communications between end user and LP node is via pubkey only. Also the networking code we use does not tag the packets with the originating IP address, so in order for IP <-> pubkey to be correlated, the attacker will need to compromise a notary node or compromise a router that is in the path of the notary node and your node.

While this is a possibility, it is unlikely that snoopy neighbor will be able to achieve this. However, do not assume it has any sort of perfect privacy. Best to assume it is somewhat public and to use JUMBLR to achieve privacy after you have DEX’ed and obtained the crypto you want.

Before you ask, JUMBLR is not yet, but it is planned for this year along with PANGEA and getting the other assetchain services online. I have a feeling, along way to a full pangea solution, I will create some decentralized gaming infrastructure, so there could well be other reference apps that feed into the BET asset. I also plan to run a “house” tradebot whose trading profits will flow through the BOTS assetchain and BOTS also gets a percentage of the DEX fees. MGW also gets a cut of the DEX fees. For HODL and SHARK which are just holding other cryptos, I think it makes sense to have a market maker that provides liquidity at close to the NAV. the CRYPTO asset is getting the notarization fees and will also get pax fees. REVS then gets a percentage of several of these assetchains, but SUPERNET is what will get a much larger percentage of most of these.

It has been a long time in development, but with KMD, all of the higher level services can now be implemented securely and so this year will be about getting those paying out dividends and from there we can continue to work toward increasing the revenues across the entire spectrum.

For those of you that have followed me for all the three years I am in crypto, you might have noticed that the last months that I am less productive than usual. I have had a personal issue that has caused this, but now it is mostly solved, so I hope to get back to full productivity soon. Luckily the timing of this was when I wasnt the critical path for the project, so there was little ill effect from it to the schedule. However, I dont expect to have to go back to the 16+ hours per day/7 days per week level as the SuperNET project is now much larger than just the core code. I will do my best to not be on the critical path and continue to complete the SuperNET vision.


Bitcointalk topic entries: 1, 2

New Feature: A Dividend API

In order to do the REVS snapshot, I decided to just implement a dividends API for iguana. There are now 2 new API calls “snapshot” and “dividends”. “snapshot” is passed in the symbol of the coin and the height for the snapshot, the height must be a multiple of bundle size and the utxoaddrs.<height> file must already exist in the DB/<symbol> folder. It will if you have been maintaining a coin in RT mode. The snapshot API just returns a complete set of { “address”, <amount> } in an array

The dividends API uses internally the snapshot API. it also specifies symbol and height, but there is also a “vals” object, which specifies how to process the snapshot array:

  • “exclude”:[] is an array of addresses to exclude from the dividends processing
  • “dust”:<float> is the smallest amount of dividend that will be sent,it is not the address balance, but rather the calculated dividend amount. Set “dust”:0 to send dividends for any non-zero value.
  • “dividend”:<float> is the total amount that is allocated across all the non-excluded addresses. Using this value and the total of address balances of non-excluded addresses allows a ratio to be calculated. This ratio is applied to the address balances to calculate the dividend amount for each address. If it is less than or equal to the dust amount, it wont be sent. If “dividend” is 0, then it means to use a 1:1 ratio
  • “prefix” and “suffix” are arbitrary strings that are used to construct the action line: <prefix> <address> <amount> <suffix>. prefix/suffix is just the simplest way I came up with to do the REVS snapshot, I can see a need for a more flexible template for the action line, but I prefer to do such things on an as needed basis.
  • “system”:1 will invoke the action line in a system call

Putting it all together:
curl –url “http://127.0.0.1:7778” –data “{\”agent\”:\”iguana\”,\”method\”:\”dividends\”,\”height\”:1486000,\”symbol\”:\”BTCD\”,\”vals\”:{\”exclude\”:[\”RXL3YXG2ceaB6C5hfJcN4fvmLH2C34knhA\”],\”dust\”:0,\”dividend\”:0,\”system\”:0,\”prefix\”:\”fiat/revs sendtoaddress\”,\”suffix\”:\”\”}}”

The above displays the sendtoaddress lines that would be invoked for the REVS snapshot.

What does this mean?
It means that iguana can now issue dividends in any currency (literally) that can run on your system and base the allocation on any iguana supported coin. With clever use of the prefix/suffix commands, maybe even non-dividend actions can be done based on stake of any iguana supported coin


Bitcointalk Topic Entry

Development History: From a Testnet to Mainnet

In a galaxy a long time ago (September)
komodo testnet was launched
dPow was added, testnet was able to survive
round robin mining was added, testnet exploded, but we were able to repair the damage.
PAX was added, after a furious battle against would be hardfork issues, testnet was able to survive with consensus intact
DEX functionality was then added, without any issues as it is really in a layer above the blockchain itself

Basically I have been able to evolve komodo during these months, adding all the features and avoiding any fatal hardfork that required a blockchain reset. A lot of blockchain resyncs by all the notary operators, but we have over 150,000 blocks in the KMD chain.

So the testnet becomes the mainnet with 99.99% probability at this point.

I believe this makes KMD the longest running zcash protocol coin. And the coins have been issued already but we dont know where to distribute them yet. There are also 44 assetchains that are also running. But since the GUI is only on github and not officially released, it seems that these facts are given little weight.


We are down to the last bugs…

The latest one found by karasugoi was an iguana compatibility error, where an iguana created tx didnt accrue interest. He also found what I think might be one of the last remaining PAX issues, which was doing multiple paxwithdraws in the same block.

This weekend I aim to get a successful DEX swap between two coins using only the basilisk mode, which means two nodes can DEX trade with each other, without even having either blockchain local.

Currently there are three modes for a coin: normal coind, iguana full node, basilisk node that uses notary nodes remotely.

This means there are 9 combinations of DEX trades. I have already gotten several of these combinations tested, but the most demanding is the basilisk <-> basilisk DEX trade, especially due to the potential of being spoofed. I just added spoof prevention and now the basilisk mode is responding virtually instantly.

you can see for yourself using any of the dex* tests scripts in SuperNET/iguana/tests from the github.com/jl777/SuperNET repo.

what this means is that for the following coins, you can DEX against any of the others without having any blockchain locally:

[“KMD”, “BTC”, “USD”, “EUR”, “JPY”, “GBP”, “AUD”, “CAD”, “CHF”, “NZD”, “CNY”, “RUB”, “MXN”, “BRL”, “INR”, “HKD”, “TRY”, “ZAR”, “PLN”, “NOK”, “SEK”, “DKK”, “CZK”, “HUF”, “ILS”, “KRW”, “MYR”, “PHP”, “RON”, “SGD”, “THB”, “BGN”, “IDR”, “HRK”, “REVS”, “SUPERNET”, “DEX”, “PANGEA”, “JUMBLR”, “BET”, “CRYPTO”, “HODL”, “SHARK”, “BOTS”, “MGW”]

LP nodes can be configured with a standard bid/ask spread for any trading pair, or it can also be configured to automatically quote prices based on the current central exchange price plus profit margin. Just like komodo doesnt try to replace bitcoin, but uses bitcoin’s security, iguana native coin DEX doesnt try to replace central exchanges, but rather uses their liquidity.

add to the list above, all the coind coins that iguana supports and also add all the coins iguana directly supports. I think pretty much immediately, komodo/iguana native DEX will have one of the widest selection of coins


Bitcointalk Topic Entry 1, 2

Tradebots Update – Predicting the Future Price

Dev branch for iguana now has most of the initial LP node code active. It currently generates realtime raw features in the SVM folder and there is an API command to generate an SVM. The SVM will then make predictions as to the future price direction.

There are currently 8 SVM’s for different timeframes, from 10 minutes to 12 hours in the future. You can configure things to track a large number of trading pairs and it even detects arbitrages that are possible if two or more exchanges are active. See tests/arbinit script.

The realtime predictors are not yet active, but SVM generation is averaging 85%+ accuracy for today’s BTC price action. A very very small sample and as we get more and more different type of days this accuracy will reduce, however the more data there is the more accurate the overall realtime forward predictions will be. It calculates prices for all BTC pairs into USD and CNY. It also monitors the NXT blockchain for the various asset prices and maps them to BTC prices, so things are all setup for automated LP node quotes for all the assetchains, plus any supported BTC trading pair.

it really is quite a lot of new code and some pretty fancy AI level predictor generation as SVM have much better performance than neural networks for large scale datasets.


Update were written to #tradebots on SuperNET Slack.

Last Few Weeks of Development Until Launch

you might have heard about the xmr html GUI having a security flaw that allowed an attacker to do any RPC command if the user went to a site with some special code.

Since iguana GUI is html based, this concerned me and I started investigating. linked up with ca333 who knows a ton about this sort of thing. What I learned was that in html space things are quite dangerous. For the convenience you have a much much much bigger attack surface.

Needless to say we had to improve the security for iguana GUI (if you are using any HTML based GUI it is advisable to make sure it is as secure as required). We finally figured out how to make the iguana html GUI as secure as a QT wallet. It did require some major additions to the baseline, but when security of your money is at stake, then we do what we have to do.

I just released the enhancements needed to the iguana core, so shortly the GUI will be able to utilize it and we can get an html wallet that is both easy to install/use and secure.

Now if your computer is infected with a keylogger or something like that, there isnt much that anything can do, even though with our approach it is somewhat resistant to keylogger. however a keylogger would open a different attack vector which might or might not be prevented based on usage. So, your system must be clean of keyloggers and such malware. In that case even if you browse to an injection attack site you will be protected. Generally speaking, best to not be browing to random sites when using an html wallet. However, if a banner ad leads to a page with injection codes…

The komodod side is in the final stretch. I rebased to the zcash bugfix version that improves performance and fixes some important protected mode bugs.

lots of progress on the assetchains, with over 40 chains now part of the komodo family. just a few more things to validate and I am actually now coding the DEX layer which will be a layer on top of the komodod and iguana core, so it is a nice flexible layer that will be relatively smooth to deploy, ie. just run it as a dapp

I thank everybody for their patience and maturity as we make improvements to the ICO site to accomodate BTCD holders. I know BTC peoples dont really care about such things, but please know that the tech is in the finalization process and this time is needed to get rid of the final showstopper issues.

I would say that a potential security breach via injection websites is pretty important to fix!

Assuming we can get the final two pax functionalities validated, we can mainnet with pax enabled. A fallback release would disable pax for the initial release and we would need to hardfork to enable it. Since the GUI for pax is not yet, it might not be a big deal either way. Personally I prefer to get all the hardforks out of the way while it is in “testnet” status.

In the next weeks you will see a lot of campaigning from the notary nodes operators and this shows that komodo is unique with a community responsive set of notary operators, instead of miners that are just silently mining. The proactive notary operators can and will bring in marketing for komodo in addition to helping expand the overall reach of komodo. Few people really understand the full scope of what komodo can do and the notary node operators are most of this few.


Bitcointalk Topic Entry

Decentralized Broadcasting Service Powered By Notary Nodes

I coded up a low level broadcast service utilizing the notary nodes. Still need to debug it, but it shouldn’t take too long. DEX will use this to propagate order-books.

It uses a combination of scaling protocols to create a generalized broadcast, where from any node you can submit a packet and all the other nodes get a copy of it. It is load balanced so as not to overload any single point and with all the notary nodes having decent hardware and bandwidth, the max capacity would be limited only by available bandwidth.

This is pretty cool, but creates a problem because it is so easy to invoke. Namely spam, which must be prevented.

I am planning to use one of the assetchain coins as the currency needed to pay for such broadcasting. Since most all of the SuperNET services will be using this, I think it makes sense to denominate the broadcast fees in SuperNET assetchain coins. With a cost of 0.0001 coins, the cost is minimal, but still enough to prevent spam.

The way things are structured, the payments of the SuperNET assetchain coin would go to the crypto777 account so it is automatically marked as a special tx and the current codebase can add support for this with minimal extra work.

I am still open to feedback on this as I haven’t coded it yet. The downside is that we will probably need some sort of SuperNET faucet to onboard new users, but all the micropayments will be fully automated like I did with the notary nodes. That codebase can be reused for the micropayments.


I had a bit of spare time, I created 10 more assetchains:
./komodod -pubkey=$pubkey -ac_name=SUPERNET -ac_supply=816061
./komodod -pubkey=$pubkey -ac_name=DEX -ac_supply=1000000
./komodod -pubkey=$pubkey -ac_name=PANGEA -ac_supply=1000000
./komodod -pubkey=$pubkey -ac_name=JUMBLR -ac_supply=1000000
./komodod -pubkey=$pubkey -ac_name=BET -ac_supply=1000000
./komodod -pubkey=$pubkey -ac_name=CRYPTO -ac_supply=1000000
./komodod -pubkey=$pubkey -ac_name=HODL -ac_supply=10000000
./komodod -pubkey=$pubkey -ac_name=SHARK -ac_supply=1400
./komodod -pubkey=$pubkey -ac_name=BOTS -ac_supply=10000000
./komodod -pubkey=$pubkey -ac_name=MGW -ac_supply=10000000

These are 1:1 issued to match the corresponding NXT Assets:

SuperNET, InstantDEX, Pangea, NXTprivacy, Privatebet, crypto777, jl777hodl, sharkfund0, NXTcoinsco and MGW

These are all zcash forks with on-demand block generation and dPoW enabled. When we get the MGW setup to use assetchain coins, there will be an automated process to convert between the assetchain coins and the NXT assets using MGW


Bitcointalk Entries: 1, 2

PAX: the First Implementation Of A Decentralized Gateway

I just released a pre-release candidate version of komodod. All of the identified hardfork changes are active, including the assetchains decentralized gateway with pricefeed.

I added anti-tampering to the pricefeed, and extended the notarization data to include approvals for withdraws. Additionally, there are limits on the total amount of each fiat that can be created. For now it is set to 10000 to 40000 fiat units per fiat. This limits the total worst case exposure to the blockchain to a small enough level. with pax2, I will activate methods that allow increasing the total allowed while maintaining the max exposure, but first step is to get the fiat cryptos in active use.

I also use the implied KMD price from BTCD conversion in the paxprice calculation. I know there is a strictly limited amount of BTCD available at below ICO prices, but it is what it is and as long as it is trading, whatever price it calculates to is a least more tangible than random guesses. Of course, with the limited liquidity, we get the silly result that marketcap goes up 10% with each 30BTC, +40% with 90 BTC and 333% gained with 150BTC.

So dont take the implied price as any stable way to calculate marketcap, it is just a very rough estimate subject to limited liquidity

I have submitted a draft of the PAX technical description a while ago, but I didnt describe the details of the decentralized gateway, even in that document as there were some issues I had to solve during the implementation. Now that it is complete, I am able to accurately describe it.

I wont go into all the details about the price feed as I already wrote that in the initial document. For here, just assume that we have a reasonable stream of prices coming into the blockchain. By reasonable, I mean that more than 51% are valid prices.

In order to calculate the KMD fiat price conversion we need to know 3 prices: KMDBTC, BTCUSD, fiatUSD (for USD only 2 prices are needed). I use BTC and USD as reference currencies as they are the de facto reference currencies used in the markets.

For the KMDBTC and BTCUSD prices, a method similar to augur is used to get a deterministic but random majority correlated price for each height. This is then price smoothed over the prior day (with a pseudo-random starting point in a circular buffer). Then the same process is done with the derived fiatKMD price, so in order to game the system, the majority of pricefeed samples need to be controlled, which requires control of a majority of the notary nodes. Due to the four-way geographic split and wide voter base for the notary nodes, we dont expect that it is practical to get majority control of the notaries.

However, as a precaution there is a max limit for each fiat allowed, so in the worst case the damage is limited. Until the dynamic rebalancing of pax2 is active, this will be sufficient to prevent any runaway scenarios.

So we can assume that for each height, we have a deterministic price consensus that is not gameable for all 32 fiats. I believe Komodo is the first to achieve this. But this is just the start of PAX! In order for PAX to work, the pricefeed needs to be converted to blockchain enforced coins. The general concept of how to do this was introduced with the multigateway (MGW) that I did almost 3 years ago, but going from a distributed MGW to a decentralized gateway required some complex steps.

Namely 5 steps.
1) paxdeposit: this is the process of creating new fiat coins dynamically by converting the corresponding amount of KMD into the fiat coin. This is achieved by burning the KMD in an OP_RETURN, which includes the specifics on prices, fiat, etc.

The KMD side is straightforward, the trick is how to convert that OP_RETURN burn into an issue of the fiat coins in the fiat assetchain. Before we can even think about doing this, we need to solve how on earth can many blockchains know the state of the other blockchains!? It is not practical to use the RPC due to performance problems when dealing with dozens of chains.

I ended up having to create a special komodostate file that records each event in a file. This allows all instances to parse the komodostates of all the other chains and thereby reproduce the identical state in all 33 chains. It was a lot harder than it sounds to get it all working stable.

2) Issue: So, given that, the fiat chain can “see” the OP_RETURN burn and when it does, it tracks which ones have been issued. Things happens all out of order, so we also need to know when we are all caught up to realtime states in both chains as we dont want to do any duplicates. It is the miner that actually does the query for any deposits without the matching issue. All the miners on all the nodes see the same pending deposits (we are in consensus after all), so regardless of which node mines the next block, the pending deposit is issued.

The way I issue the fiat coin is to extend the coinbase transaction with extra outputs, along with a summary OP_RETURN. When this is seen (on all nodes for all chains), the deposit is marked as completed so it wont be duplicated.

Now the fiat coins are issued at the appropriate price, once and only once. Note that the same method can be used to create a sidechain, just remove the pricefeed and do it 1:1

Believe it or not, this is the easy side of the decentralized gateway. The reason is that all assetchains have to run komodod, so we can be assured that all nodes running EUR chain has access to the realtime komodod state. However, the reverse is not true. Not all komodod instances will have access to the current (or any) fiat chain instance, since a user can pick and choose which assetchains to run.

This assymmetry creates quite a lot of potential issues. Once approach is to use the segwit method of having two classes of nodes, the updated segwit ones that are able to validate transactions and the other nodes that just need to trust the segwit nodes to validate. With enough nodes running to verify things, it might be acceptable, but we have notary nodes, so I decided to enhance that level of security with a notarization of each withdraw. So there is a withdraw, approval, redeem on the conversion of fiat coin back to KMD.

3) paxwithdraw: nearly identical to the paxdeposit, just done on the fiat chain side to create the OP_RETURN burn of the fiat coins.

4) approval: notary nodes add paxwithdraw details to the notarized data. All nodes that run both komodod and the fiat chain are able to verify that the paxwithdraw details in the notarized data are valid. This means we get the segwit level validation enhanced with notary node consensus on the withdraw data. Now even a node that is only running komodod gets almost as reliable data as actually running all the fiat chains.

5) redeem: all nodes that are mining gets the pending withdraws (using paxpending RPC) so whichever node mines the next block, the redeem is issued properly. Once and only once. All the nodes are able to validate using the approval data or the data directly from the fiat chain.

One final thing which we shouldnt forget is to make sure that prior to 2) issue, that it is notarized. this way fiat coins are issued only after they are notarized in btc chain. On the return side, since we are piggy backing on a notarization, we get the benefit of notary node consensus for the redeem.

Hopefully the above answers all the questions you have had about how on earth can pax possibly work.

Next up is to verify the iguana notary dapp is updated to match the komodod. I think it already is, but need to verify it. iguana notary dapp changes can be made without a hardfork, so it is less priority than komodod changes that would create a hardfork.

The pre-release candidate isnt a release candidate as I am waiting for zcash to release 1.04 so I can rebase to it. That is expected next week, which is good timing as it will likely take a week or so to verify the assetchains.

With fixing any bugs in the above as priority, I will be shifting to porting the DEX to the notary nodes and getting it integrated into komodod so iguana and komodod can cooperatively DEX. This wont be hardfork changes to komodod, so it can happen even after the initial mainnet version is out, but I prefer to get it done prior to it.

I am ready to do REVS snapshot at any moment, but due to popular demand to allow people to participate in the snapshot without any local wallet, we are updating the ICO site to allow for this and this takes time. I am not personally working on it, so please dont ask me its status.

I believe the current PAX is the first implementation of a decentralized gateway, especially one with an external price feed. By using the notary nodes, it becomes possible to do things that otherwise would not be possible to do. Komodo is thus able to continue to be enhanced with what otherwise would require centralized servers, but instead of that, Komodo can use the notaries + decentralized methods.

PAX proves that it can be done, but it is just the first of such a hybrid functionality to be active.


Bitcointalk Topic Entry

Unexpected Difficulty Explosion Drill

Yesterday we dealt with a diff bomb!

It was a self-inflicted one due to a small problem with the mining algo, but that combined with a few missing notary nodes to almost stop the blocks.

Diff peaked at over 10 billion.

The good news is this was a nice fire fighting drill to see how well komodo responds to such an event. No matter how much you prepare, the unexpected will always arrive unexpectedly. So it is good to be able to respond.

And respond we did. I was already a bit nervous about the rapidly rising diff and had made a corrective release, but we needed a much more forceful way to reduce the diff. I also solved how to avoid such a diff explosion (and implosion) in the future and the good news for non-notary miners is that at least 1 block in 64 will be available for normal mining. Additionally any miner that is not online to mine their blocks will lose it to the non-notary miners. Though the notaries can also do non-notary mining.

I am not sure what percentage of notaries will be offline at any given moment, but now there is a specific financial penalty of the lost KMD to provide a direct feedback to incentivize being online 100% of the time.

By having at least 1 block in 65 go to external miners, the diff will be tied to the actual hashrate. It might sound easy to create a dual-diff mechanism that allows notary nodes to mine at easy diff, while still allowing non-notary nodes to mine at normal diff, with a seamless transition between the two.

Its actually quite tricky to avoid all the edge cases and I have had to evolve the algo to arrive at the current solution. Initially it was a literal round-robin where for a given block only one notary was able to mine it at low diff. This was not bad, but it affected the block production whenever a notary node is not online. Also, the lowdiff makes it trivial to find a block and without a delay, we would end up with a really fast block.

There is another issue that needs to be avoided and that is a mining war between notary nodes. While this is not the end of the world, the financial model is affected as the mined KMD is expected to be a significant part of the notary’s revenues and we are wanting to minimize the BTC subsidy needed. If real mining resources are needed on top of everything else, each notary’s financial equation changes, not to mention the reallocation of the precious KMD.

Ideally, we can rely on the notary nodes to not compete as they are a team, but considering the massive controversy over what region some border countries belong to, I would feel better if there was a technical enforcement of the round robin.

Now this timer is what created the diff explosion. I had it set to 45 seconds, figuring that it was close to 60 seconds and at the time it was before the ratification of the 64 notaries, so there was a lot of non-notary blocks, thus minimizing the faster block issue. Granted if all notaries changed their timer to 30 seconds or less for whatever reason, this would have been an issue.

Then the ratification of the 64 notaries got into the blockchain and activated.

At that point the vast majority of blocks came in at 45 seconds to 50 seconds, which triggers a diff increase to slow down the blocks. Of course, the notary exemption doesnt care what the actual diff is, so we ended up in a never ending increase of the diff. Which by itself is not such and issue as if a notary is going to mine a block at low diff, what does it matter? It matters when the designated notary node is not there and it would take 20 million CPUs to mine the next block!

I had enhanced the set of eligible notary nodes for low diff to be the chosen node plus any notary that didnt generate any of the previous 63 blocks. Which worked quite well until we got to where all active notaries had generated a block recently and the current chosen notary was missing.

After I released a few hotfixes, we ended up not able to tell which was the right mainchain as the blocks were so slow and you cant resolve a fork on the block it forks. Anyway, after a notary wide fire drill to update to the hotfix version, we got the blocks coming in regularly again and the diff is down over 1000x and is currently ~3 million. Another day and it should get down to a reasonable mineable level which is required for proper operation of the system.

The current post-82000 height solution to this is as follows:

A) At any given height, any notary that hasnt generated a block in the prior 65 blocks is eligible for low diff.

B) After a block is mined, a minimum of 59 seconds from start of mining is ensured before submitting to the network, with another 0 to 2 seconds randomly added.

I believe these two simple rules achieves all the desired properties, including avoiding a mining war between notary nodes and preventing artificial speeding up of blocks.

There is little to gain from reducing the 59 second delay as after a block is mined, a notary needs to wait for all the others to mine a block and also the one external. So all reducing the time would do is create one fast block. And if many notaries somehow all decide to do this, we have the diff explosion to disincentivize such fiddling. What point to generate blocks a bit faster if it just creates a diff explosion event. Also, it will be easily seen on the blockchain that a specific notary has adjusted the setting, so it is not anything that would go unnoticed.

Blocks will come in like clockwork, literally, as most all the blocks are based on the clock 59 seconds + random(2 seconds) for average of 60 seconds. Currently this is not active, but already it can be seen that blocks are quite regular: http://www.zpool.ca/explorer/1720

This method is also very resilient from a missing notary condition as it does not matter which notary, just any notary that doesnt have a recent (within 65 blocks) mined block. which means if any such notary is available, a new block arrives. After the initial race as to who wins the intial blocks, we end up in a randomized round robin as at any given block, all notaries have already mined a recent block and have to wait for the single eligible node to mine its block.

And here is where the external (non-notary) mined block comes into play to calibrate the diff to reality. Whenever all notaries are waiting, which would be (1 + inactive notaries) blocks per 65, the block is up for grabs for external miners and ineligible notaries.

It is a good sign that by simplifying things it achieves more. If anybody sees any issues, please post! There might well be a way to improve it more, but at least the current solution guarantees blocks to online notaries, gets us clockwork blocks, calibrates diff to reality and penalizes offline notaries.

My assessment is that Komodo during pre-mainnet has responded to a blockchain event as well or better than most mainnet coins and kudos to all the notary operators!

On the coding side, I am down to a couple of final issues dealing with the fiat chains and once that is done the current mainchain can become the official mainchain. I estimate 95%+ chance that the current “testnet” chain will become the actual mainnet, which means all test coins mined are the real KMD


Bitcointalk Topic Entry