The AtomicDEX application uses it's backend API served by daemon named
marketmaker. The earlier version was written in C language by jl777, the
marketmaker version 1. This new
marketmaker version 2, in short
mm2, which is the backend API service for the graphical application such as AtomicDEX is built from scratch in modern and popular programming, Rust language.
I had some doubts and questions of my own too, so I asked jl777 in #ask-jl777 channel in Komodo Discord. Sharing that chat log here. It has better answers.
grewalsatinder Today at 8:14 PM
@jl777c mm2 uses libtorrent to create it's independent network to communicate between nodes, where the orderbook and ordermatching is done.
with new network ID, it is also scalable.
So, in that sense why we needed another scalable solution such as dexp2p?
what difference is the main difference in relation to mm2 scaling in comparison to dexp2p ?
Trying to understand the reason why dexp2p was started. :slight_smile:
jl777c Today at 8:15 PM
libtorrent is slow
grewalsatinder Today at 8:16 PM
isn't the komodod also using libtorrent lib in it's core for p2p layer? :thinking:
if not, what is it using? and how it's faster?
jl777c Today at 8:16 PM
mm2 used a very inefficient method to broadcast as it was just a placeholder, so it needed a new p2p network layer. after seeing discussions about it for 6 months, i decided to just write it as i knew i could in a few weeks
bitcoin doesnt use libtorrent that i know of
i use the same peer networking as bitcoin code, it is the actual same code that is handling a new p2p message
its faster as i made it faster
I had thought that there would already be an easy to use module to spawn a p2p network that would synchronize any sort of data that was…
grewalsatinder Today at 8:18 PM
okay, from bitcoin whitepaper I thought it's still using libtorrent for p2p.
is that some custom p2p or anything I can read on what bitcoin's p2p layer is built upon?
jl777c Today at 8:19 PM
i think it is libevent, just tcp connections
and its p2p messages
grewalsatinder Today at 8:19 PM
oh! that's why it's faster!
jl777c Today at 8:20 PM
direct peer to peer dedicated tcp/ip connections
send a message directly to a peer
given that, and only that, to get an efficient broadcast without a lot of redundant traffic and having it decent performance...
well it did take me a few weeks to get working, maybe someone else would have taken a lot longer to achieve the current level of performance and efficiency
basically the mm2 orderbook matching bogged down with 100+ users, but it wasnt coded to handle more than test amounts of traffic. someone had to make a more efficient orderbook/ordermatching, so i did. not sure what the mystery is, unless you missed the atomicDEX stress test
that is the origin of dexp2p, but i made it generic so it can be used for any usecase that needs to have efficient message broadcasting over p2p network, so it supports file sync, even video streaming. with the tagA/tagB and e2e encrypted messages it can easily be used by a dapp to implement a notarizer or a discord equivalent, or subatomic, or ...
blockchains are non-realtime and have an unpredictable blocktime, 10 minute wait for notarization, basically not usable for realtime cases. dexp2p provides the realtime response and it is inside the coin daemon, so it can do subatomics (dexp2p directly linked to payments)
grewalsatinder Today at 8:27 PM
I missed the tests I think. I was busy somewhere else, and was only glancing here on channels for some months occasionally.
jl777c Today at 8:28 PM
it basically melted down during the first minutes
so clearly a solution was needed. maybe i overdid it and made a new highly flexible decentralized messaging solution, but it works quite well for orderbook, ordermatching and trade executions
grewalsatinder Today at 8:33 PM
clears things much better :thumbsup:
can you also share any specific challenges faced stabilising dexp2p to handle big data packets on p2p network?
jl777c Today at 8:35 PM
i implemented a txpow on a per packet basis, the larger the packet the more txpow needed for more data, so it gets very expensive to send big packets
the file transfers use 10000 byte packets and it is reassembled into slices and slices into file
once the low level messaging worked reliably, the file level was just a couple days work, most of it fiddling with getting streaming working. no real hurdles
though there is a bug still with streaming, so maybe that is the hurdle
grewalsatinder Today at 8:38 PM
wow! so the txpow, which was used in DiceCC, and I think also used in AdoptedPoW also helped with dexp2p... :thinking:
jl777c Today at 8:39 PM
it is a different variant of txpow
but yes, the concept of using pow per message is quite useful, but bitmessage did this years ago
grewalsatinder Today at 8:39 PM
ah yes.... with bitmessage it clears more :slight_smile:
jl777c Today at 8:40 PM
there is also an anonsend rpc call that sends to everybody
grewalsatinder Today at 8:40 PM
need bit more to know on txpow
is that also like bitcoin's PoW? using SHA256 to do PoW per transaction or event message sending data packet over network?
jl777c Today at 8:41 PM
but only the destination can decrypt it
every data packet gets a hash sha256(curve25519(sha256(message)))
the curve25519 is to make it fpga resistant and also there are no ASIC for the above hash that i know of and unlikely for there to be anytime soon
grewalsatinder Today at 8:44 PM
and I guess it won't be a big deal to change it, since it's jus p2p network based and nothing related to blockchain and avoiding hardfork concerns, in case some naughty fpga programmer figures it out and make that algo work on fpga. Please correct me if any assumption in this line is wrong.
jl777c Today at 8:47 PM
there are "network hardforks" that would be needed if the basic algos are changed, but that is a matter of updating to new version, there is no blockchain to sync. also, i designed it so that dapps can track if their messages are getting through, and if not, just raise the priority level. that wont help against an fpga, but against a spam attack, it can make the spam exponentially more expensive to make
it also allows someone who really, really wants to broadcast their packet to generate a higher priority one. when the network is saturated, it goes into a different mode and routes the highest priority packets first, the lowest ones it might just totally ignore
jl777cToday at 8:54 PM
every coin can get its own -dexp2p network within its set of peers, just add the -dexp2p=2