Exploring ZeroNet for Fun
Introduction
Over the past month, I've been exploring ZeroNet for fun. ZeroNet is a fully distributed peer to peer network. It's an open source python software project. It's a mix of bitcoin blockchain for authentication and the BitTorrent protocol for transferring files and data between your node and others. It uses the Namecoin .bit private top level domains for domain names. Your client is a server and can start cloning and serving up sites from your first browse of a site. The site is served up by any clients which have also browsed the site and cloned it. These are your peers and your servers.The ZeroNet author wrote up a good presentation to give you a high-level understanding of it.
However, I'm interested in one level deeper of that content. What does ZeroNet actually do on the wire? Well, here's what ZeroNet 0.4.1-rc works from a network perspective. Let's explore the bootstrap process.
Bootstrap
First up, it boots up, checks for optional Tor connectivity, and then contacts a few bittorrent trackers to seed itself.
trackers=['zero://boot3rdez4rzn36x.onion:15441', 'zero://boot.zeronet.io#f36ca555bee6ba216b14d10f38c16f7769ff064e0e37d887603548cc2e64191d:15441', 'udp://tracker.coppersurfer.tk:6969', 'udp://tracker.leechers-paradise.org:6969', 'udp://9.rarbg.com:2710', 'http://tracker.aletorrenty.pl:2710/announce', 'http://explodie.org:6969/announce', 'http://tracker1.wasabii.com.tw:6969/announce']
It then tests for an existing Tor connection, if the default Tor control port exists, it'll try to use that across localhost, if not it starts up a bundled copy of tor.exe (if on Windows). If unix-like system, then it dies at trying to find tor on its control port.
TorManager Connecting to 127.0.0.1:9051 ERROR TorManager Tor controller connect error: error: [Errno 10061] [Error 10061] No connection could be made because the target machine actively refused it. in TorManager.py line 154 > _socket2.py line 228 DEBUG TorManager Tor proxy port 127.0.0.1:9050 check error: No connection INFO TorManager Starting Tor client tools/tor/tor.exe... DEBUG TorManager Connecting to 127.0.0.1:49051 DEBUG TorManager > PROTOCOLINFO DEBUG TorManager < 250-PROTOCOLINFO 1 250-AUTH METHODS=COOKIE,SAFECOOKIE COOKIEFILE="\\ZeroNet\\tools\\tor\\data\\control_auth_cookie" 250-VERSION Tor="0.2.8.7" 250 OK DEBUG TorManager > AUTHENTICATE c7b0e9c651f617d1033a148412457286dd12beaeb25965a4bad05e01009d59f4 DEBUG TorManager < 250 OK
It then loads the sites in its cache.
DEBUG SiteManager Sites not loaded yet...
DEBUG SiteManager Loading sites...
DEBUG SiteManager SiteManager added 27 sites
DEBUG SiteManager Updated merger sites in 0.124s
In my case, I have 27 sites cached/serving by default.
A webUI server is started on TCP port 43110.
INFO - Starting servers....
INFO Ui.UiServer --------------------------------------
INFO Ui.UiServer Web interface: http://127.0.0.1:43110/
INFO Ui.UiServer --------------------------------------
If a private key doesn't already exist, it creates a new 2048-bit RSA key.
DEBUG - Generating RSA cert and key PEM files...Loading 'screen' into random state - done
Generating a 2048 bit RSA private key
................................+++
.......+++
unable to write 'random state'
writing new private key to 'data/key-rsa.pem'
And then starts up a file server on TCP port 15441. If that port isn't already open, it tries to open it to the Internet. It also checks a 3rd party domain to see if this domain can reach your port from outside your network. If this all fails, and you have tor available, you create a number of hidden services to be your own file server. I'm aware there's a private key here, but I just ran this for demo purposes. For each service you have cloned locally, you are going to create a hidden service if port 15441 isn't available from outside. This is 27 .onion sites in my case.
DEBUG FileServer Binding to: *:15441, (msgpack: 0.4.8), supported crypt: ['tls-rsa'] INFO FileServer [BAD :(] Port closed: Port 15441 is closed. INFO FileServer Trying to open port using UpnpPunch... DEBUG - Trying to open port 15441. INFO FileServer Checking port 15441 using portchecker.co... ERROR FileServer UpnpPunch run error: UpnpError: Failed to communicate with igd using port 15441 on local machine after 3 tries. in FileServer.py line 73 > UpnpPunch.py line 319 > UpnpPunch.py line 310 DEBUG TorManager Start onions DEBUG TorManager > ADD_ONION NEW:RSA1024 port=15441 DEBUG TorManager < 250-ServiceID=6xe4ay5uwuq2ccmv 250-PrivateKey=RSA1024:MIICWwIBAAKBgQDBnUf6xCLzPSCbskZy2oZtYOSGHhuIejK6Dr0kFSTeK7Apfw/ojwNe71F2x5lv7TdyIzd+aBj2D4/w7X+G1sTZ2oWTsdk4fTsE/1GLq33Mc91wedVtYa7vZxHfZvWCrp5beeZf44UZFxkAoXRXQlqPXTFfC8DZOSeYqWlOVlnFoQIDAQABAoGASpnztigE33xaKCPVCUQyL8r5wsOvEDMlgJNVRaXwArsKsbKHyq/wOJA173KRKJNandv78fTiSU1NxSN909LFHhB0QPGmUr8z/lcOjlgQlUKTiiZJe85e0qcYNyngEAgwSgdRPnLopaBAR1q6im/RYl5OAkoeZR0kHEWGsIUPy5ECQQDlYMdc/RybdToFbIT1Tyu4VSjzUgzBvJrCq6sPPfE6Em/CJuEm0+xrWEUEss0ViNkTpUwR4RgPDs+XFFHAxrSPAkEA2BXm1Svegy3prt2htmVRbD8eGqaOAMv/1IEl7P55FNel5HvBr4WoV9VBe2dXiw/lP7B7rZcYMS5Nx3wjb1TazwJAaBipoUvFJc/nivqaxHEvmDIdqX89Btc/LMWXlg04YnBng6b9Ww+mJXkjqWQzqmg2i7HieeK7dmn0T84K4//JhQJAOEn8R6uV2zYqmZLCfXEHNNt8TBT3CN4bVpAOAR6JDBTg8hQK8F4w7DBxTmQtOdx6K40dqqqhcq5NNJnx4R6JoQJAFcHS9qIPTsLJt8kgDRDg+cidTuZSpw3oXIhz61GSjuUYp3acbdb8ISbV219SzNiPTSr47PcAiVedEcYSLac8sA== 250 OK DEBUG TorManager Created new hidden service for 1LQNHchUKBBtv5cguPuy92Cy7gLnZJCySs: 6xe4ay5uwuq2ccmv
Initial Page
Remember, your client is also a server, so the Hello ZeroNet page is served locally, from your own cache. Here are the pages loading up for the local Hello page.
DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:26] "GET /1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D HTTP/1.1" 200 2878 0.009000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:26] "GET /uimedia/all.css?rev=1536 HTTP/1.1" 200 29692 0.117000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:26] "GET /1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D/?wrapper_nonce=7b8fb215dc0eb274a83aa4ced893e724b425fde23ed6040b4f0fb0ca1a184318 HTTP/1.1" 200 1267 0.002000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:26] "GET /uimedia/all.js?rev=1536 HTTP/1.1" 200 167948 0.008000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:26] "GET /1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D/css/all.css HTTP/1.1" 200 180893 0.006000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:26] "GET /1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D/js/all.js HTTP/1.1" 200 117167 0.005000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:26] "GET /1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D/img/logo.png HTTP/1.1" 200 1986 0.001000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:27] "GET /1HeLLo4uzjaLetFx6NH3PMwFP3qbRbTf3D/img/logo_big.png HTTP/1.1" 200 11666 0.001000 DEBUG Ui.UiServer 127.0.0.1 - - [2016-10-16 01:10:28] "GET /Websocket?wrapper_key=1e5ef269f38e192311f18f6431c3a89226900d4e389ab54d2a924eb9462b2dc6 HTTP/1.1" 101 129 1.531000
This is close to Common Log Format that most webserver log parsers can handle it and show you who and what is loading up your site.
When you click on a zite to visit, here's what happens:
DEBUG WorkerManager:12h51u..WWtv Added worker: 185.15.72.63:15441, workers: 1/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 95.84.130.202:15441, workers: 2/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 119.0.231.162:15441, workers: 3/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 83.134.43.35:15441, workers: 4/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 95.183.50.208:15441, workers: 5/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 139.162.248.13:15441, workers: 6/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 106.185.35.18:15441, workers: 7/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 188.166.30.172:15441, workers: 8/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 81.61.174.149:15441, workers: 9/10 DEBUG WorkerManager:12h51u..WWtv Added worker: 5.80.22.86:15441, workers: 10/10 DEBUG Site:12h51u..WWtv Found 24 peers, new: 15, total: 15 DEBUG FileServer Conn# 1 185.15.72.63 [?] > Connecting... DEBUG FileServer Conn# 2 95.84.130.202 [?] > Connecting... DEBUG FileServer Conn# 3 119.0.231.162 [?] > Connecting... DEBUG FileServer Conn# 4 83.134.43.35 [?] > Connecting...
Pretty straightforward connections to other peers. You also contact a tracker to find more peers for the requested site. Of course, the tracker may or may not respond.
DEBUG Site:1UDbAD..MkoV Http tracker http://tracker.aletorrenty.pl:2710/announce?uploaded=0&downloaded=0&numwant=30&compact=1&event=started&peer_id=-ZN0041-j5jH8LXDVgqI&port=0&info_hash=%18%AE%CE%26%B8w%18%04%89%DB%B7%C0d%FE%A1%FFJ%FD%D5%C3&left=0 error: HTTP Error 502: No data received from server or forwarder ERROR Site:1UDbAD..MkoV Announce to 0 trackers in 1.495s, failed
Sometimes a site or a peer is only available via tor. Remember the hidden service you created? This is likely a machine behind a NAT or firewall which blocks port 15441 from inbound connections, so zeronet uses tor hidden services to bypass the blockage.
DEBUG FileServer Conn#15 boot3rdez4rzn36x.onion [?] > Connecting... DEBUG TorManager Creating new socket to boot3rdez4rzn36x.onion:15441
And then zeronet spends time trying to keep your sites up to date by swapping content.json files around to mirror them locally. And sometimes a site goes offline or the peers don't have a valid content.json file.
DEBUG Site:1Mr5rX..Y5Ba Try to get listModifications from peers: [, , , , , , , , , ] since: 1472343086.44
DEBUG FileServer Conn#106 92.233.100.212 [v2] > Crypt out connection using: tls-rsa (server side: False)...
ERROR Site:1MaiL5..Ju27 data/users/1L4CuqRerzY4w8iRvcUUMJFRhzWz9FvZ8E/content.json this file is archived!
By now, you may realize this is all from debug log files, not from the actual network as mentioned at the beginning. You may be correct.
Here's a flow graph of loading pages.
And here's the I/O graph of the network traffic from boot to loading a few pages.
Next post I'll talk about some of the sites found, what they look like on the wire, and how the communication looks while my zeronet client/server is just idling, but serving up parts of sites I have in local cache.