Browsed by
Tag: dapp

Ethereum developer Working environment

Ethereum developer Working environment

If you want to create smart contracts and dApps using Ethereum, you first must have a working environment. Here I’ll share with you my personal setup and why I choose to use it.

The big picture

Our working environment should contain three main components:

Local Machine

Keep it as light as possible

1.       Google Chrome (or Chromium for Linux users)

2.       MetaMask and/or Ethereum node of your choice

3.       Optional – SSH client and terminal

Digital Ocean Droplet

This is your real working environment. Where your code will run

System requirements:

·       Ubuntu 16.04 x64

·       1 CPU

·       1 GB RAM

·       1 GB SWAP

We’ll install:

·       NodeJS

·       NPM

·       truffle

·       testRpc

·       Optional – Chai

·       Optional – Meteor

C9 IDE/ SSH Terminal

 

Register to C9 IDE. We’ll only use it as a browser based terminal with SSH capacities.

 

This is how our working environment looks like:

Creating the Digital Ocean Droplet:

  1. Create your digital ocean account. Use the following link to get 10 USD voucher. Pay attention; the subscription is auto renewing. If by the end of the first month you don’t want to be charged again, you should manually change your billing settings.
  2. Once logged in press Create -> Droplets. Choose the one that has:
    1 CPU 64 bits
    1 GB RAM
    Ubuntu 16.04 X64
  1. Name your droplet and press the Create button
  2. After few minutes you should get an email with your Droplet IP address and root password.

Connecting to your Droplet using the c9.io terminal and SSH (Optional – can also be achieved using a locally installed SSH client and terminal):

  1. Create a free account at c9.io
  2. SSH into your droplet by typing ssh root@YourIPAddress
  3. When you first log into your droplet, you’ll be asked to change your root

Set your Digital Ocean Droplet:

Create SWAP file 1 GB of RAM isn’t enough.
sudo fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
Add SUDO user Don’t always work as root!
adduser shlomi sudo
Change your user account
su shlomi
cd /home/sholmi
Install NodeJS and NPM This is the framework on which we’ll develop our app.
curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo apt-get install -y build-essential
sudo apt-get install npm
Install Truffle and TestRPC
npm install -g truffle
npm install -g testrpc

The relation between Truffle and testRPC:

testRPC creates a mock Ethereum blockchain. It saves you a lot of time when testing your smart contract. Because you’re the owner of the blockchain, you can mine new blocks instantly. Usually, when using Ethereum, you’ll have to wait for new blocks to be mined whenever you check you smart contract and while block time is much faster in Ethereum than it’s in Bitcoin, it might still be very distracting to have a test case that contains more than 2-5 interaction with the blockchain.

Another great benefit of using testRPC is the fact that it immediately creates multiple accounts with balances so that you can test the use of your smart contract by multiple users.

TestRPC TIP:

When you first deploy testrpc you’ll be giving a mnemonic 12 words phrase. If you write this phrase down, you can later re-deploy your restRPC with the same accounts.

Working with truffle:

Create a working folder and initialize your truffle project
mkdir myProject
cd myProject
truffle init
Make sure testRPC runs in the background
testrc -m "tortoise fall alarm push dream proof
 broccoli size draft betray view gather"
Test your project
truffle migrate
truffle test

Tip for metamask and testRPC:

We can set metaMask to work with our own private testRPC node simply by adding it into our custom rpc list. Open metaMask and change to custom RPC. Then add
http://yourDropletIPAddress:8545  //You might need to open port 8545
And voila, Now metaMask is set to work with your own private blockchain!
Get your Bitcoin address using Ethereum smart contract

Get your Bitcoin address using Ethereum smart contract

Ethereum and Bitcoin are both using the same type of encryption, the ECC (Elliptic Curve Cryptography) over the same graph (256k1). While it’s not really recommended, the same key pair can be used both for Bitcoin and Ethereum.

A simple Solidity code can be used to get the Bitcoin address of a public key. Such a code can run locally (as a constant function) on the Ethereum Virtual Machine to save gas, or as a regular Ethereum transaction.

The code in this example requires the user to insert their public key in its uncompressed format as an input; then it produces the binary address that matches that uncompressed public key for the main Bitcoin network. The code can be easily modified to work with compressed public keys as well (just remove the yPoint and add the side of the ECC graph). The code can also be amended to give the binary address of other testnet/namecoin.

 

How to create a Bitcoin address

The most basic process of deriving Bitcoin address from a public key is set in the following technical documentation.

 As you can clearly see, there’s very little to it than just hashing and appending.

Recreating the process in Solidity

First, let’s generate a random keypair using bitaddress.org. Under the tab “wallet details” we can see the uncompressed public key.

The public key
xPoint = C4BB8E42F7DA5504A456C16BE533549DA4FE580279382478F3365FF7CCBF032D
yPoint = 68A73547E809F1ABFAA51D10019E8AC682D1205448042326E9E3B91841CB9FA7

Now let’s create our smart contract in Solidity:

pragma solidity ^0.4.11;

contract BitValid{
	
	bytes32 constant mask4 = 0xffffffff00000000000000000000000000000000000000000000000000000000;
	bytes1 constant network = 0x00;


	function getBitcoinAddress(
			bytes32 _xPoint,
			bytes32 _yPoint)
			constant
			returns(
				bytes20 hashedPubKey,
				bytes4 checkSum,
				bytes1 network)
	{
		hashedPubKey 	= getHashedPublicKey(_xPoint, _yPoint);
 		checkSum 	= getCheckSum(hashedPubKey);
 		network 	= network;
	}

	function getHashedPublicKey(
			bytes32 _xPoint,
			bytes32 _yPoint)
			constant
			returns(
				bytes20 hashedPubKey)
	{
		var startingByte = 0x04;
 		return ripemd160(sha256(startingByte, _xPoint, _yPoint));
	}

	function getCheckSum(
			bytes20 _hashedPubKey)
			constant
			returns(
				bytes4 checkSum)
	{
		var full = sha256((sha256(network, _hashedPubKey)));
		return bytes4(full&mask4);
	}
}

The function getBitocinAddress() takes the x and y coordinate of the public key from the user, both are 32 bytes long (the uncompressed public key) and will return 3 variables, the hashed public key (bytes20), the checksum (bytes4) and the network starting byte (bytes1).

The network starting byte is currently hard codded to 0x00 (the main starting code). You can change this code to work with any other test network.

The hashed public key is obtained by hashing the public key (both x and y coordinates) with the starting byte 0x04 twice (as described in the technical documentation). Once with sha256 and then again with ripemd160. The finale result is 20 bytes long.
function getHashedPublicKey(
		bytes32 _xPoint,
		bytes32 _yPoint)
		constant
		returns(
			bytes20 hashedPubKey)
{
	var startingByte = 0x04;
	return ripemd160(sha256(startingByte, _xPoint, _yPoint));
}
After we got the hashed public key, we’ll prepend the network byte to it and hash it again twice using the sha256 function. The result of 32 bytes long is used to construct the checksum, a special 4 bytes that are used to allow another user to verify that the Bitcoin address they’re sending to is indeed a valid address.
bytes32 constant mask4 = 0xffffffff00000000000000000000000000000000000000000000000000000000;

function getCheckSum(
		bytes20 _hashedPubKey)
		constant
		returns(
			bytes4 checkSum)
{
	var full = sha256((sha256(network, _hashedPubKey)));
	return bytes4(full&mask4);
}
We don’t need all of the 32 bytes, only the first 4 bytes, but slicing variables is a hard thing to do in Solidity. Luckily, Solidity does allow for easy bit manipulation and masking. You’ll need to create a mask of 32 bytes to match the 32 bytes of the sha256 output. This mask should take only the first 4 bytes, as they’re the real checksum.
The full result (32 bytes) = 0x4c30ed507a508af52063560ff8f1c09e66be0587868a0b8ca21ab337440e4e8e
Mask for the first 4 bytes = 0xffffffff00000000000000000000000000000000000000000000000000000000
checksum = 0x4c30ed50

The results

At the end of the day, we have the following three components to return to the user, the network byte (currently hard coded), the hashed public key and the checksum. These are the three components that make up a Bitcoin address.

However, this isn’t the last step. In Bitcoin, a special type of encoding is used called base58. The current code doesn’t convert the result into base58 (I’ll leave it for another day), so we’ll be forced to do this step manually.

The following website provides some tools to convert our bytecode into base58. This is basically the final Bitcoin address.

At the end of the day

Using Solidity to retrieve the Bitcoin address that matches a specific public key (and therefore, a private key as well) might be useful when you’re trying to create a smart contract that maps some events between entities on both blockchains and I suspect might have some value when dealing with identities. The procedure isn’t cheap on gas but can be done locally using the EVM. It’s a shame that there’s no access to the bytecode of the transactions in Solidity since it could have made the process of finding the Bitcoin address of the message sender automated.

Proof of data integrity – Solidity code

Proof of data integrity – Solidity code

Update:

I’ve simplified to code to take more advantage of the boolean data type Ethereum offers.

Now the mapping is (bytes32=>bool) instead of (bytes32=>bytes32).

The boolean array is used to prove the existence of a single document. The root of the tree is stored once and is hashed again with the new input.

The complete code can be found on Github. The old test files can be used on this code as well

struct tree{
    bytes32 root;
    mapping(bytes32=>bool) dataExist;
}

bytes32 public empty;

mapping (address=>tree) public users;    
****

function addData(
			uint256 _input,
			address _user)
			returns (bool success)
{
	var data   = keccak256(_input);        
	var oldRoot = getUserRoot(_user);
    var newRoot = hashTheTwo(data, oldRoot);

    users[_user].dataExist[data] = true;

    users[_user].root = newRoot;

    return true;
}
****

function checkDataIntegrity
        (uint256[] _data, 
        address _user)
        constant         
        returns (bool complete){ 

	var oldRoot = empty;                         
    
    for (uint i = 0; i < _data.length; i++) {     
        var data = keccak256(_data[i]);
        if(users[_user].dataExist[data]){
            var root = hashTheTwo(data, oldRoot);
            oldRoot = root;
            continue;             
        }else{
            return false;
        }
    }

    if(root == getUserRoot(_user)){
    	return true;
    }else{
    	return false;
    }
}
Contract: test 01
    The first stage is Deploying Data
      ✓ Deploys the Data contract
      ✓ Register account 0 user (69ms)
    Adds three datas to account 0 user tree
      ✓ get account 0 user root - should be undefined (79ms)
      ✓ Adds the first data 1 (108ms)
      ✓ Adds the second data 2 (86ms)
      ✓ Adds the third data 5 (79ms)
    Check data integrity
      ✓ Pass the complete array [1, 2, 5] - expect true (122ms)
      ✓ Pass the  array [1, 2] - expect false (93ms)
      ✓ Pass the  array [1, 5, 2] - expect false (83ms)


  9 passing (818ms)
    

Storing data on the blockchain

Current blockchain architecture allows us to decentralized valuable information. The most obvious example is the blockchain itself, which is nothing more than just a database that each user can interact with. The users can download a copy of the blockchain, parse it and extract any information that is meaningful to them, to add information to that database, to check its integrity and so on.

But adding information directly to the blockchain is a problematic process. For one, it’s highly expensive. Whether you’re using Bitcoin or Ethereum as the blockchain on which to store your data, you’ll soon find out that any attempt to save more than a few bytes of data at a time can get ridiculously expensive. For that reason, many have started to use the blockchain as a method to “proof existence” of said document. In this process, instead of publishing the full document on the blockchain, the document is hashed using a prespecified hashing algorithm. This practice means that the owner of the document uses the blockchain not as a mean to store his or hers document, but to prove:

  1. Ownership over the said document (As long as he keep the private key from which the transaction was deployed)
  2. The existence of the said document at a specific point in time (by looking at the block header timestamp)
  3. The integrity of that specific document, as each minor change to the original file, will result in an entirely different hash.

By keeping the document yourself, you’re also able to better handle your privacy, as now instead of publishing your own private documents on a public blockchain for all to see, you’re only posting the result of a hash function, which is extremely difficult to Didact the contains of the original document from.

Such a system might be sufficient for sporadic use. But what happens if we want to create a system that new documents are continuously added to it. And we want to be able to prove the integrity of each individual document, both by itself and in conjunction to those preceding it?

 

Binary trees.

Binary trees are not a new thing in blockchains. Merkle trees and roots are used in Bitcoin and Ethereum to store and organize transactions and to allow for merged mining. In Ethereum the trees are also used to access the storage (variables) and states of the blockchain.

One of the great characteristics of binary trees is the ability to use them, plus some hashing algorithm to prove the integrity of the data stored in it.

Let’s have a look at the most common example, the Bitcoin Merkle tree. In this tree, each leaf represents one transaction. These transactions are hashed together again and again until finally, the final hash (the root) is produced. Storing the root require much less space than when storing all of the transactions data. But if I want to check that a specific transaction is indeed a part of a specific block, I can reconstruct that said block Merkle root by myself. In this case, that means that instead of storing all of the transactions that took place in the blockchain, I maintain only copies of the transaction that is relevant to me (and usually it requires less than a half of the transactions in a block).

In Bitcoin this tree is used both to proof the integrity of the block and to make it easier to validate transaction without having a full copy of the blockchain

 

My proposal 

Knowing the advantages of binary trees, hashing, key encryptions, and filled with the motivation to create a user specific database that will allow him/her to maintain control over his/hers private information, while still being able to prove their ownership over the information and the integrity of that information, I decided to play a little with different Solidity codes. The idea was to use mapping as the mean of creating pairs of leaves and root.

 

Each leaf is hashed with the previous root to produce the new root of the tree

Each root is the hashed of all of the chain that lies bellow its level, plus the new leaf added. This way, each attached leaf if linked and chained with the rest.

pragma solidity ^0.4.6;

contract Data{
    
    struct tree{
        bytes32 root;
        mapping(bytes32=>bytes32) leafAndRoot;
    }

    bytes32 public empty;                                           // Hard codded         
    
    mapping (address=>tree) public users;
    
    function newUser(){                                             // To do - Modifer "onlyNewUser" 
        users[msg.sender];
    }
    
    function addData(
            uint256 _data,              // To do - serialize data/non empty
            address _user)
            returns (bool success){  
        
        var leaf    = keccak256(_data);   // Hashing the input
        var oldRoot = getUserRoot(_user);
        var newRoot = hashTheTwo(leaf, oldRoot);
        
        users[_user].leafAndRoot[leaf] = newRoot;
        users[_user].root = newRoot;

        return true;
    }
    
    function getRoot(
            uint256 _leafData,      // The input is in plain uint256 and hashed format to allow for future UI to be devloped
            address _user)
            constant 
            returns (bytes32 root){ // The root of specific leaf
        
        var leaf = keccak256(_leafData);                                            // Hashing the input
        return users[_user].leafAndRoot[leaf];
    }
    
    function getUserRoot(
                address _user)
                constant
                returns (bytes32 root){ // The higest (last) root      
        return users[_user].root;
    }

    function hashTheTwo
                (bytes32 _a, // To do - serialize data/non empty
                bytes32 _b)  // To do - serialize data/non empty
                constant
                returns (bytes32 hashed){         
        return keccak256(_a, _b);
    }
    
    function checkDataIntegrity
            (uint256[] _data, // To do - serialize data/non empty
            address _user)
            constant         // Run localy
            returns (bool complete){ 
         
         

        var oldRoot = empty;                         // Hard codded                        
        for (uint i = 0; i < _data.length; i++) {    // Reconstructing the tree     
            var data = keccak256(_data[i]);          // Hashing the input
            var root = hashTheTwo(data, oldRoot);
            
            if(root == getRoot(_data[i], _user)){         
                oldRoot = root;
                continue;
            }else{
                return false;
            }
        }        

        if (oldRoot == getUserRoot(_user)){
            return true;
        }else{
            return false;
        }
    }
}

For each new user, a new struct object is created containing two parts, The latest root in the tree, and the tree itself. The tree maps from bytes32 (the data/leaf) to the bytes32 of the root. That way a user can look up for a specific information and, if the root is valid, attest that the said information is indeed present in the database, while others cannot tell what the real information is just by looking at the blockchain.

Currently, due to input limitations in solidity, the easiest way to input and parse and the array is by using u/int array. Future implementation might include bytes32[] array or even direct string array as input.

All the input values are hashed to get a uniform 32 bytes result and to increase privacy.

function hashTheTwo(
        bytes32 _a,
        bytes32 _b)
        constant
        returns (bytes32){         // To do - serialize data/non empty
    return keccak256(_a, _b);
}

The data is then hashed together with the highest existing root to receive the new root of the tree (If the tree is empty, meaning no root exist yet, the first leaf is hashed with empty bytes32 variable).

The new root is then stored in a dedicated variable to allow adding extra information without manually looking for the latest existing root.

function addData
        (uint256 _data,
        address _user)
        returns (bool){                 // To do - serialize data/non empty
    
    var leaf    = keccak256(_data);     // Hashing the input
    var oldRoot = getUserRoot(_user);
    var newRoot = hashTheTwo(leaf, oldRoot);

    users[_user].leafAndRoot[leaf] = newRoot;
    users[_user].root = newRoot;

    return true;
}

When trying to prove the authenticity of a single entry, it’s enough to just check for the existence of a (none empty) root that corresponds to that specific piece of information.

function getRoot(
            uint256 _leafData,
            address _user)
            constant
            returns (bytes32){ // The root of specific leaf

    var leaf = keccak256(_leafData);                                            // Hashing the input
    return users[_user].leafAndRoot[leaf];
}

// Can also be rewtiren to give bool result

function isExist(
            uint256 _leafData,
            address _user)
            constant
            returns (bool exist){
            
    var leaf = keccak256(_leafData);                                            // Hashing the input
    if(getRoot(_leafData, _user) != 0x00){
        return true;
    }else{
        return false;
    }            
}

Proving the existence of the entire database is done by providing all the pieces in their proper order and reconstructing the finale root. If the results match the one stored on the blockchain, that means that the owner of that data array has a complete copy of that array.

function checkDataIntegrity(
            uint256[] _data,
            address _user)
            constant
            returns (bool){ 
     
     // To do - serialize data/non empty
     // Run localy

    var oldRoot = empty;    // Hard codded                        
    for (uint i = 0; i < _data.length; i++) {

        var data = keccak256(_data[i]);         // Hashing the input
        var root = hashTheTwo(data, oldRoot);

        if(root == getRoot(_data[i], _user)){   // Reconstructing the tree 
            oldRoot = root;
            continue;
        }else{
            return false;
        }
    }        

    if (oldRoot == getUserRoot(_user)){
        return true;
    }else{
        return false;
    }
}

The complete code plus test file can be found on my github page

Contract: test 01
    The first stage is Deploying Data
      ✓ Deploys the Data contract
      ✓ Register account 0 user (69ms)
    Adds three datas to account 0 user tree
      ✓ get account 0 user root - should be undefined (79ms)
      ✓ Adds the first data 1 (108ms)
      ✓ Adds the second data 2 (86ms)
      ✓ Adds the third data 5 (79ms)
    Check data integrity
      ✓ Pass the complete array [1, 2, 5] - expect true (122ms)
      ✓ Pass the  array [1, 2] - expect false (93ms)
      ✓ Pass the  array [1, 5, 2] - expect false (83ms)


  9 passing (818ms)
    

What can I do with it?

I can use the above system to prove that I’m in control over my own data, that I have the original data and that I maintain a complete copy of my database. Such a system can be combined with other types of encryptions to prove that the data is both belongs to me, complete, and recognized by other authorities.

I prove that I have a my complete medical file, I prove that the said message belongs to me and that it was signed by myself and by the doctor
Ethereum signature validation app

Ethereum signature validation app

Import: This article is for educational purposes only. Don’t attempt to incorporate the codes and methods presented here into working applications and don’t use keys that are associated with your real Bitcoin/Ethereum wallets.

 

The key pair

Key pair (Asymmetric encryption) is one of the building blocks of current blockchain solutions and cryptocurrencies, without it, Bitcoin, Ethereum and other blockchains were not possible.

The idea behind this tool is quite simple: Encrypting information using one key (public key) and decryption it using another (private key).

This short video gives a great introduction to the concept of key pairs as well as an explanation to the mathematical background behind RSA asymmetric encryption

  • Rememebr that both Bitcoin and Ethereum aren’t using RSA encryption. Instead they’re using ECC (Elliptic Curve). The mathematical background is different for the two, yet the main principle is the same.

 

As seen in the video, asymmetric encryption has been around for quite some time and it’s by no mean a unique feature of the blockchain. However, both Bitcoin and Ethereum (and probably many other blockchains) utilize it in a slightly different way. Rather than using the public key to encrypt a message, they’re using the private key to sign a message.

This signed message has some interesting proprieties, but the one thing what makes it really useful in the blockchain context is that the public key can be used to validate to authenticity of the signer.

 

original_msg = "hello"

private_key = "0x010203..."

public_key = "0x0f0e0d..."

signed_message = sign(original_msg, private_key) = "0xaabbcc..."

validate(public_key, original_msg) = True

As you can see, the idea wasn’t necessarily to hide the information (the original message need to be presented in order to validate authenticity of the signer). Instead, we use this method to prove the owner of a specific private key is indeed the one who signed the original message.

In the blockchain sense, Bob can sign the original_msg -the transaction (which is of course publicly available to anyone who have a copy of the blockchain), and by providing his own public key and the signature, everyone can verify that that specific message was indeed signed by Bob.

 

The validator

Originally, I planned to write some basics codes demonstrating the process in Bitcoin and Ethereum, but while studying Ethereum more in depth, I encountered the Solidity ecrecover method that returns the address associated with the signed message, and I immediately sat down to create the Validator, a simple app that uses web3.js to sign a message at the client side, and then uses smart contract to get the address of the signer (btw, the ability to display the address of the one who signed the message hints at another interesting property which I might go deeper into in another post).

The source code can be found here:
https://github.com/Shultzi/validator

Step by step

The process was very simple, first I created the smart contract:

contract Validator{
    
    function constVerify(bytes32 r, bytes32 s, uint8 v, bytes32 hash) constant returns(address) {
        return ecrecover(hash, v, r, s);    
    
    function verify(bytes32 r, bytes32 s, uint8 v, bytes32 hash) returns(address) {
        return ecrecover(hash, v, r, s);
    } 
}

The contract Validator contains two functions but both are basically doing the same. The only different is that the first one is constant, that means no transaction is sent to the Ethereum network (caution! request might still be sent to a remote node if you don’t run a local Ethereum node!). This function will instantly return the address of the one who signed the original message. The other function is not a constant function, that means that a transaction will be sent to the Ethereum blockchain and the returned result will be verified by all of the users (consider the implications in terms of privacy!) the result however will not be immediately displayed to the end user – instead, in my app the user will receive the hash of the transaction. The user can then look it up on the blockchain.

The ecrecover function itself is very simple to use, all you need is the hash of the original message (hash) and the signed message (r, s, v).

The original message is hashed to ensure that uniform size, so that regardless to the size of the original message, we’ll always have a hash variable of exactly 32 bytes.

The r, s, v are all parameters of the signed message. The signed message itself (as you might already saw in the above video) is actually a combination of 3 variables.

full_sign = 0x042995e2dd996f8d234be59a623f3a2b02d3fb91187f48eaf563723b342225cc16599133550d998c880ecb1a8d29f47216f0397e30e415b95d92490f3b4ca6201b

r = 042995e2dd996f8d234be59a623f3a2b02d3fb91187f48eaf563723b342225cc //32 bytes

s = 16599133550d998c880ecb1a8d29f47216f0397e30e415b95d92490f3b4ca620 //32 bytes

v = 1b //uint8 (1 byte)

The signed message can be received using the web3.js library. I used meteor (based on nodejs) to launch my application.

Once it was launched, I declared web3 object like so:

if(typeof web3 !== 'undefined'){
  web3 = new Web3(web3.currentProvider);
}else{
	web3 = new Web3(new Web3.providers.HttpProvider("http://localhost:8080"));
};

The web3 is connected to metamask via chrome extension, but you can of course use your own preferred client like geth, parity or testrpc.

Once web3 is declared, getting the full signature is a very simple thing to do:

web3.eth.sign(web3.eth.accounts[0], web3.sha3(msgToSign.value), function(err, res){signedmsg = res;});

This is the full signature. r + s + v. We’ll need to break it into their component. Just remember that:

  • The first 32 bytes are the r value
  • The second 32 bytes are the s value
  • The last byte is v value (uint8)

You can read more about signature structure here

Breaking the signature into its r, s, v values is a fairly easy process that can be done with the following JavaScript code.

r = "0x" + signedMsg.value.slice(2, 66); //Treated as hex
s = "0x" + signedMsg.value.slice(66, 130); //treated as hex
v = new Buffer(signedMsg.value.slice(130, 132), "hex"); // we care for the numeric value. The Ethereum function expects uint8 and not hex.
v = v[0].valueOf();
h = web3.sha3(originalMsg.value); //we hash the original message to keep it as 32 bytes, regardless to the input size.

Now the only thing that is remained is to send these values along with the original message to smart contract, and get the result back.

Validator.verify(r, s, v, h, function(err,res){         		
    Template.address.set("The transaction id is: " + res);
});

Validator.constVerify(r, s, v, h, function(err,res){
    Template.address.set("The signer address: " + res);
});

The final result

http://nobelgoeshere.com/ (The site isn’t secured. Don’t sign anything of value!)

Signing and validating message in ethereum

 

Mixing environments – Creating working environment for blockchain developers

Mixing environments – Creating working environment for blockchain developers

This article is part of a series of articles depicting my experience with creating and conducting an 8 week long blockchain app development course in Brazil.

 

What tools should be used when teaching blockchain

 

The term blockchain is often misused. Very rarely do people use the term blockchain to describe anything beyond a chain of blocks. A lot of the time when people talk about the blockchain and its application, they basically refer to a somewhat wide variety of technologies, architectures, tools and protocols that, once properly combined and implemented, creates that “blockchain” they are referring to.

When I created the course, it was obvious to me that in order to properly teach the students how to work with “the blockchain”, I’ll first need to spend a lot time dealing with many different technologies and tools. There isn’t just one blockchain IDE or concept to examine; rather there are quite a number of them. Take key pair for example; private and public keys are some of the most crucial (and known) features in many crypto-currencies and blockchains, but they are by no means specific to blockchains. Many people use key pairs off chain. The same holds for many concepts that are highly integrated into the common view about blockchains – Hashing functions, signatures (and keys), scripts and stack architecture, byzantine general problem, bytes codes, merkle trees, DAGs and more.

Each feature in the list above represent another tool/approach/use case/concept that stands by itself but is also crucial to creating what is commonly known as “the blockchain”. This fact posed a great challenge for me when I tried to create the course. It was obvious to me that the course is aimed at people who want to learn how to develop their own blockchain applications and solutions, which meant that it will require the students to get their hands somewhat dirty in codes, command line prompt, and different computational tools.

The challenge here lay in choosing the right tools to work with while remembering that each item on the list should be taught in a manner that is  adequate on the one hand, but without going to a level too deep and insignificant for the course on the other hand. It was also important that there should be a clear difference in the relations between the different and individual items.  I knew I wasn’t hired to teach the students how to program or how to work with different environments. However, making the assumption that they had adequate programming knowledge, enough not to require any introduction to that programing language/ environment/ tools seemed quite optimistic at best, and downright stupid at worst. This is even more so when dealing with a variety of different tools and languages.

I decided to do my best to choose the most user friendly working environments – even at the cost of efficiency and future usability.

Numerous developers have their own working environment. However, I was convinced that every code, example and CLI command/tool should be properly tested and documented in a single uniform environment. The last thing I wanted to do was stand in front of the class while in the background, my code failed to compile. The result of this is that I tried  a lot of different environments while always keeping in mind that the environment to be used should fulfill the following requirements;

  1. It needs to support all the tools I require that my students use.
  2. It shouldn’t affect in anyway the students’ computers, working environments, file systems, paths and/or jeopardizes their computer security in any way.
  3. It should be uniform for all the students.
  4. It should be easy and fast to set and reset whenever needed.
  5. It should be as user friendly as possible.

 

After a few experimentations, I decided to work with the following configurations:

 

  1. Cloud9 level 1 IDE environment with the following installations:
    1. Python-pip.
    2. Python-virtualenv.
    3. Virtual environments for Python 2.7 and 3.5
    4. Ethereum SOLC
    5. Tcpdump (for some reasons, not all c9 workspaces had it installed)
    6. The following pip packages (base58, ecdsa)
Cloud 9 was used for running python files and as a uniform terminal.
  1. Digital ocean Ubuntu 16.041 X64 droplet with the following installations:
    1. Nodejs 6
    2. Meteor Javascript framework version 1.3.4 with web3 and bitcore-lib packages.
    3. The following changes were optional for a few students:
      1. Installing ipfs and running ipfs daemon and adding ipfs-api package to their meteor app. (For those who wished to work with IPFS).
      2. Adding swap file of 4 gb. (For those with memory issues).
  • Use openssh. (More IDE flexibility for advanced users).

 

  1. Solidity browser compiler was mostly used for writing and deploying smart contracts. SOLC (installed on c9) was used by a few students who required some more advanced contracts (mostly when containing libraries).

 

  1. The only 2 components the students were required to install on their own machines were:
    1. Chrome/Chromium with metamask addon.
    2. Wireshark.

 

Cloud9 provided a well-tested and easy to configure working environment that was consistent for all students. It was used mainly to run the Python codes the students created, to compile some Solidity codes (using SOLC), to catch some packets using tcpdump (The tcpdump files were later downloaded and examined using wireshark) and to access digital ocean droplet using ssh.

I was very pleased with this working environment as it was quite robust, highly configurable, not local and easy to reset – Basically it was a great playground to get dirty with, without having to worry about damaging the students’ native environment.

 

Digital ocean droplets were used to provide the students with a uniform platform on which they can create their apps. Meteor is a well-documented JavaScript framework. It was obvious to me that if the students were expected to create applications, they should also have access to some JavaScript tools as both Bitcoin and Ethereum have some very powerful tools for app developers – mainly web3 for Ethereum and Bitcore for Bitcoin.

There’s also another npm package for compiling Solidity (similar to SOLC), but unfortunately, I’ve experienced a lot of compatibility issues with that package and decided to ban the students from using it. IPFS-api is another useful tool for more advanced students who are interested in working with IPFS.

It is important to note that although I did discuss IPFS with some students, I didn’t consider it an important part of the course. First, the system is still in a very early stage. Secondly, the main goal of the course was to teach the students how to develop blockchain applications, and not necessarily decentralized applications (although the two might have a lot in common, they’re not mutually the same) and IPFS just didn’t really fit the slot. Besides, I already had an ample amount of topics to focus on and teach my students (And I must admit; I’m not that much of an expert in this platform myself).

Another point to consider is that in a future course, in the case where there’s no promise to create apps, digital ocean might still be used. In this case, JavaScript libraries can be taught by using clean nodeJS interface.

 

Metamask and solidity browser were wonderful and very easy to use tools. In a manner of minutes, the student had yet another playground to play with Solidity and the Ethereum blockchain.

(It’s important to note that I took some time to make sure ALL of the students were using clean metamask installation WITHOUT any of their real wallets imported to it and only on the Ropsten testnet).

 

One last note about truffle

I also feel compelled to justify a little further my decision to exclude the use of truffle and/or embark (with testrpc) during the course and instead choosing to work with solidity browser compiler. The thing is,  at the time, both truffle and embark had some memory issues that forced me to use another swap file (both when tested on Cloud9 and when tested on digital ocean droplet). In addition to that,  most smart contracts required were easy to deploy from the Solidity web compiler. For specific ad hoc contracts that required the use of a more robust compiler, Ethereum SOLC was used on cloud 9 (SOLC didn’t had any memory issues). I do however recognize that truffle and embark are major tools in the industry and I’m defiantly planning to integrate them into future courses.

Ethereum dApp front end. Display variables.

Ethereum dApp front end. Display variables.

Catching up

Few weeks ago I created a video tutorial describing the basics of creating Ethereum decentralized application (dApp). I created the app in the tutorial by using meteor, web3.js add-on for meteor and metamask. The video can be found here:

Now it’s time to expend on the previous video and see other ways to extracts more advanced information out of the blockchain.

The smart contract

For this example I’ve created a simple Ethereum smart contact. This smart contract contains one public string variable called name and one function called changeName:

contract demo{
    
    string public name = "Shlomi";
    
    function changeName(string _newName){
        name = _newName;
    }
}

I’ve transmitted this contract to the Modern test-net using metamask, and the contract can now be found under the address: 0x53a95ce8fcdd9080ba7c26711207e45382ae4180

Solidity creates get_variable functions for public variables!

A nice feature in Solidity is the ability to call public variables as if they are functions. The result of the function call can then be transferred to another callback function, e.g:

myContract.name(function(err, res){
  alert(res)
})

This is highly useful when dealing with web development as it allows for an easy way to interact with light clients (such as metamask) and helps to provide a better user experience (We can, for example define more user friendly error messages in case we can’t properly interact with the blockchain). In our code, once the “function” name is called, the web3 object myContract will look for a variable called name that is stored at the blockchain at address 0x53a9... Then the result (In this case, my name Shlomi) will be passed to the anonymous callback function and then presented in a pop-up window.

And this is how you set the web3 object

(In the previous video I explained how to install meteor with the Ethereum web3 package)

In order to interact with smart contract we usually need 3 parameters.

  1. The address of the smart contract we want to interact with. You can get the address for your own contract by scanning the blockchain for your own Ethereum address and the list of transactions associated with it.
  2. The Interface or ABIarray. I usually prefer to refer to this parameter simply as ABI, so from here on it will be simply called ABI.

    In the web3 documentation it's called ABIarray. In the the solidiy web compiler it's called Interface.
    In the web3 documentation it’s called ABIarray. In the the solidiy web compiler it’s called Interface.
  3. The bytecode or data. I prefer to refer to this parameter as Data. So from here on it will be simply called Data.

 

The ABI and Data are usually provided by the compiler you’re working with. (I’m using solidity-browser It’s a great tool for beginners!)
Now we can declare the web3 object myContract. Using two of the three parameters: The contract address and the contract ABI (The data will be used later).

contractAddress = "0x53a95ce8fcdd9080ba7c26711207e45382ae4180"

ABI = [{"constant":true,"inputs":[],"name":"name","outputs":[{"name":"","type":"string"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"_newName","type":"string"}],"name":"changeName","outputs":[],"payable":false,"type":"function"}]

Frozeman:template-var. Easy way to display variables from callback functions.

This part actually has nothing to do with Ethereum, but because we’re going to use many (many) callback functions, it’s always nice to have an easy way to display the result in a more html friendly manner. Template-var gives us the option to set variables directly from within the callback function.

First we need to add it to our meteor project. The package can be found here.

Now what we need to do is to initialize the template object. In our helpers methods we’ll add the following code:

var template = Template.instance();

Once our template was declared, we can simply use it in our callback function.

Template.hello.helpers({
  counter() {
    var template = Template.instance();

    myContract = web3.eth.contract(ABIArray).at(contractAddress);

    myContract.name(function(err, res){
      TemplateVar.set(template, "name", res);
    })
  },
});

The first argument is the template that we’ve just declared. The second is the name of the variable, and the third is the value that we want the argument to have. In our case it’s the result of the name “function”.

One last step

In our html file, we need to call to our templateVar instance by inserting:

 
The string stored at the "name" variable is: {{counter}} {{TemplateVar.get "name"}}.

Now whenever we’ll load our app, the variable name will incorporated in our project as if it was part of the original html file.