Browsed by
Tag: bitcoin protocol

Get your Bitcoin address using Ethereum smart contract

Get your Bitcoin address using Ethereum smart contract

Ethereum and Bitcoin are both using the same type of encryption, the ECC (Elliptic Curve Cryptography) over the same graph (256k1). While it’s not really recommended, the same key pair can be used both for Bitcoin and Ethereum.

A simple Solidity code can be used to get the Bitcoin address of a public key. Such a code can run locally (as a constant function) on the Ethereum Virtual Machine to save gas, or as a regular Ethereum transaction.

The code in this example requires the user to insert their public key in its uncompressed format as an input; then it produces the binary address that matches that uncompressed public key for the main Bitcoin network. The code can be easily modified to work with compressed public keys as well (just remove the yPoint and add the side of the ECC graph). The code can also be amended to give the binary address of other testnet/namecoin.

 

How to create a Bitcoin address

The most basic process of deriving Bitcoin address from a public key is set in the following technical documentation.

 As you can clearly see, there’s very little to it than just hashing and appending.

Recreating the process in Solidity

First, let’s generate a random keypair using bitaddress.org. Under the tab “wallet details” we can see the uncompressed public key.

The public key
xPoint = C4BB8E42F7DA5504A456C16BE533549DA4FE580279382478F3365FF7CCBF032D
yPoint = 68A73547E809F1ABFAA51D10019E8AC682D1205448042326E9E3B91841CB9FA7

Now let’s create our smart contract in Solidity:

pragma solidity ^0.4.11;

contract BitValid{
	
	bytes32 constant mask4 = 0xffffffff00000000000000000000000000000000000000000000000000000000;
	bytes1 constant network = 0x00;


	function getBitcoinAddress(
			bytes32 _xPoint,
			bytes32 _yPoint)
			constant
			returns(
				bytes20 hashedPubKey,
				bytes4 checkSum,
				bytes1 network)
	{
		hashedPubKey 	= getHashedPublicKey(_xPoint, _yPoint);
 		checkSum 	= getCheckSum(hashedPubKey);
 		network 	= network;
	}

	function getHashedPublicKey(
			bytes32 _xPoint,
			bytes32 _yPoint)
			constant
			returns(
				bytes20 hashedPubKey)
	{
		var startingByte = 0x04;
 		return ripemd160(sha256(startingByte, _xPoint, _yPoint));
	}

	function getCheckSum(
			bytes20 _hashedPubKey)
			constant
			returns(
				bytes4 checkSum)
	{
		var full = sha256((sha256(network, _hashedPubKey)));
		return bytes4(full&mask4);
	}
}

The function getBitocinAddress() takes the x and y coordinate of the public key from the user, both are 32 bytes long (the uncompressed public key) and will return 3 variables, the hashed public key (bytes20), the checksum (bytes4) and the network starting byte (bytes1).

The network starting byte is currently hard codded to 0x00 (the main starting code). You can change this code to work with any other test network.

The hashed public key is obtained by hashing the public key (both x and y coordinates) with the starting byte 0x04 twice (as described in the technical documentation). Once with sha256 and then again with ripemd160. The finale result is 20 bytes long.
function getHashedPublicKey(
		bytes32 _xPoint,
		bytes32 _yPoint)
		constant
		returns(
			bytes20 hashedPubKey)
{
	var startingByte = 0x04;
	return ripemd160(sha256(startingByte, _xPoint, _yPoint));
}
After we got the hashed public key, we’ll prepend the network byte to it and hash it again twice using the sha256 function. The result of 32 bytes long is used to construct the checksum, a special 4 bytes that are used to allow another user to verify that the Bitcoin address they’re sending to is indeed a valid address.
bytes32 constant mask4 = 0xffffffff00000000000000000000000000000000000000000000000000000000;

function getCheckSum(
		bytes20 _hashedPubKey)
		constant
		returns(
			bytes4 checkSum)
{
	var full = sha256((sha256(network, _hashedPubKey)));
	return bytes4(full&mask4);
}
We don’t need all of the 32 bytes, only the first 4 bytes, but slicing variables is a hard thing to do in Solidity. Luckily, Solidity does allow for easy bit manipulation and masking. You’ll need to create a mask of 32 bytes to match the 32 bytes of the sha256 output. This mask should take only the first 4 bytes, as they’re the real checksum.
The full result (32 bytes) = 0x4c30ed507a508af52063560ff8f1c09e66be0587868a0b8ca21ab337440e4e8e
Mask for the first 4 bytes = 0xffffffff00000000000000000000000000000000000000000000000000000000
checksum = 0x4c30ed50

The results

At the end of the day, we have the following three components to return to the user, the network byte (currently hard coded), the hashed public key and the checksum. These are the three components that make up a Bitcoin address.

However, this isn’t the last step. In Bitcoin, a special type of encoding is used called base58. The current code doesn’t convert the result into base58 (I’ll leave it for another day), so we’ll be forced to do this step manually.

The following website provides some tools to convert our bytecode into base58. This is basically the final Bitcoin address.

At the end of the day

Using Solidity to retrieve the Bitcoin address that matches a specific public key (and therefore, a private key as well) might be useful when you’re trying to create a smart contract that maps some events between entities on both blockchains and I suspect might have some value when dealing with identities. The procedure isn’t cheap on gas but can be done locally using the EVM. It’s a shame that there’s no access to the bytecode of the transactions in Solidity since it could have made the process of finding the Bitcoin address of the message sender automated.

The blockchain developer path – Blockchain at Berkeley

The blockchain developer path – Blockchain at Berkeley

Mapping the blockchain education ecosystem

When I created my first tutorials almost two years ago, there were very few educational resources about blockchains. The ecosystem was sorely lacking in good courses, tutorials and guides to ease the learning curve for newcomers. I remember the many days I’ve spent reading raw codes and technical documentation until I was finally able to manually connect to the Bitcoin network, create keys, sign transactions, etc.

But two years is forever in the world of blockchain, now many companies, universities, and individuals are flooding the market, offering their services as educators and validators.

The thing is that this ecosystem is still kind of a wild west. There are very little standardization and collaboration between different players in this ecosystem. And the final result is sub-optimal, both for those who wish to educate themselves and for those who want to work with them (future employers/potential business partners).

But fear not, I’ve taken upon myself to try most of the major courses available and to receive as many certifications as possible. And I’ve got some interesting insights on the right route an aspiring blockchain developer should take.

I the following posts I’ll review the following courses and certification process:

  • Princeton University – Bitcoin and Cryptocurrency Technologies. Available as a full course at Coursera
  • Berkely University – BLOCKCHAIN at Berkely.
  • Stanford University – Bitcoin and Cryptocurrencies.
  • Multiple Youtube channels and videos (including my own)
  • Book – Mastering Bitcoin: Unlocking Digital Cryptocurrencies by Andreas Antonopoulos (1st Edition)
  • Book – Bitcoin and Cryptocurrency Technologies by Princeton University (Draft version)
  • Udemy Ethereum: Decentralized Application Design & Development
  • Diginomics – Ethereum developers course (Created by myself more than a year ago)
  • IBM – Blockchain Essentials for Developers (Includes certification)
  • C4 – Certified Bitcoin Professional.

 

 

Made with love

Blockchain at Berkeley

From the oldest to the youngest. Blockchain at Berkeley was created just short of a year by a group of (mostly) undergrads from Berkeley University. Don’t let it fool you. The fact that these students are undergrads is nothing short of a plus for this program. It’s fresh, created with love, well paced, publicly available and highly recommended.

The full program is currently only available as a course (Academic credits) at Berkeley University. The program publishes its materials online. As I never had the pleasure of participating in the live program, my review refers only to what is available online.

 

The program structure – Beginners

The program website might be somewhat confusing at first. Some of the links are circulars, and it’s hard to find the main Education page. Pressing on the Education tab will circulate you between different live programs, workshops, and resources. The DECAL tab, which contains the (live) course assignments, reading materials, and webcasts is where you should start as it organize all of the material and resources in chronological order.

It’s my view that to understand the blockchain; one must first understand Bitcoin as it is the most researched, documented, robust and stable example of the blockchain out there. Berkeley does just this; they’re making sure that the students first understand what the blockchain Is, how it was created, how it works, and what it’s not, before moving to other implementations of the blockchain (mostly Ethreum, although hyperledger and zcash are also mentioned in later lectures).

The programs begin with the history of Bitcoin and digital money, moves to a high-level view of the protocol – from the consensus point of view and then gives a brief introduction to the crypto aspects of Bitcoin by introducing ECDSA. Once ECDSA is presented the lecture starts to get somewhat technical for many casual users (ECC properties), but not too technical that it might prevent those who are committed to start and understand how the concept of signing and asymmetric encryption. That’s by far one of the best examples I’ve seen for “mitigating the knowledge gap.” The terms are technical and well explained in a slow and forgiving pace without dummying it down.

The good work doesn’t stop with ECDSA. The next lecture: “Bitcoin Mechanics and Optimization” is another great example of how to teach some technical aspects of the Bitcoin blockchain such as double spend, transactions, and scripts (only P2PKH so far), Merkle trees, UTXOs vs. Accounts (Another great little detail that sets the foundations for the advanced Ethereum part of the program). All is presented in a professional, yet chewable way.

The program also provides an excellent top down review of mining, game theory and potential attacks. All was wonderfully constructed with a top-down view of all the relevant concepts and interesting and thought-provoking examples.  This part is also used as a bridge to introduce other blockchains (mainly Ethereum that is also the star of the next lecture)

Ethereum received only one lecture in this program, and that’s a shame. I feel as if the Ethereum lecture is the weakest one in the batch, but it’s evident that this is only due to the time limitation. The top-down review of Ethereum was good for the casual user/learner, but not on par with what the program offered so far. Many aspects of the Ethereum blockchain architecture and of the VM were completely omitted from the presentation. The decision to talk about the considerations in creating smart contracts was counterproductive, and the result is a jumble of terms and concepts that are explained in too much haste. It’s important to notice that there’s another presentation on EVM that wasn’t mentioned in the lecture

Once Ethereum is out of the way, the program focusses again on Bitcoin and presents many future ideas about it: Federated chains, switching hashing algorithm, PBTF and more. I mostly enjoyed the part about payment channels and lightning network which, together with Aaron van Wirdum’s Understanding the Lightning Network on Bitcoin Magazine, is the best resource for non-coder who wants to know how LN should work.

 

All of the lectures and presentations are available online at the DECAL tab. However, it’s clear that the presentations weren’t created to be a standalone learning material. It’s almost impossible to make much sense of them without watching the video, which is weird because the videos are no more than a live recording of the instructor, reading from the board. The class noises might be somewhat distracting, and it’s a shame that it’s impossible to hear the questions from the audience.

 

The program structure – Advanced

The materials in the DECAL tab are providing an excellent review of the blockchain with a mix of technicality and causality that can appeal to a broad audience. But for the more technical students, a more technical information is required. That is the WORKSHOP tab comes to play. Over here stored the more advanced presentations. Dealing extensively with the EVM, smart contract coding (Solidity), data architecture, math, etc.

However, as mentioned before, it’s clear that the presentations weren’t meant to stand alone. Someone needs to explain them. And unfortunately, the workshop doesn’t contain any videos. That makes most of the workshop material somewhat useless for students (but an excellent resource for educators/teachers).

Another great aspect of the program is the “Whitepaper circle.” Once in a while, the students upload their technical review of a relevant whitepaper. The videos are highly recommended.

 

Interactivity

Unfortunately, it’s impossible to participate in any way in the program (unless of course, you’re currently in Berkeley). It might have been nice to have a forum/slack for the program. I’m sure many would’ve appreciated the ability to interact with the other students and instructors directly.

 

Assignments and Certification

As mentioned before, there’s no way to interact with the program. You cannot participate in any way. No assignments, no certification. Only nothing. Shame.

 

Final thoughts

Blockchain at Berkeley was created by a group of enthusiasts undergrad students – and that only means good things about the taught material. Concise, useful, up to date, and the precise blend to appeal both to highly technical and to the more general audience.

The fact that this program was created almost as an independent side project of the said students is noticeable in its presentation. The website and presentations are beautifully done but poorly integrated into a coherence experience. I understand that when something as useful as this is giving to the public for free, it’s almost rude criticizing the way it is wrapped, but I feel as is the creators really did wanted to have something useful – a place for developers from all over the world to learn the secrets of the blockchain – the love and effort is evident to see, but in order to achieve it, some work need to be put into packaging the program.

Final note – The best program out there for technical people (coding is not a must) who wants to really understand what the blockchain is. A bit hard to find your way around it – but worth it.

 

Commit yourself to this program if: ·       You’re a tech-savvy person taking is first steps into the world of blockchain AND willing to put some effort into understanding the technicality of it.
Time to complete the course: ·       Thirteen weeks. Previous classes are all available on youtube and can be watched in one go
Interaction: ·       None. Too bad
Materials:

·       YouTube channel. Contains all lecture videos and some whitepaper circle videos.

·       Presentations

·       Some code examples from the workshop

·       Their resource page is also worth checking out

Certification: ·       None. Too bad

 

The blockchain developer path – Princeton University – Bitcoin and Cryptocurrency Technologies

The blockchain developer path – Princeton University – Bitcoin and Cryptocurrency Technologies

Mapping the blockchain education ecosystem

When I created my first tutorials almost two years ago, there were very few educational resources about blockchains. The ecosystem was sorely lacking in good courses, tutorials and guides to ease the learning curve for newcomers. I remember the many days I’ve spent reading raw codes and technical documentation until I was finally able to manually connect to the Bitcoin network, create keys, sign transactions, etc.

But two years is forever in the world of blockchain, now many companies, universities, and individuals are flooding the market, offering their services as educators and validators.

The thing is that this ecosystem is still kind of a wild west. There are very little standardization and collaboration between different players in this ecosystem. And the final result is sub-optimal, both for those who wish to educate themselves and for those who want to work with them (future employers/potential business partners).

But fear not, I’ve taken upon myself to try most of the major courses available and to receive as many certifications as possible. And I’ve got some interesting insights on the right route an aspiring blockchain developer should take.

I the following posts I’ll review the following courses and certification process:

  • Princeton University – Bitcoin and Cryptocurrency Technologies. Available as a full course at Coursera
  • Berkely University – BLOCKCHAIN at Berkely.
  • Stanford University – Bitcoin and Cryptocurrencies.
  • Multiple Youtube channels and videos (including my own)
  • Book – Mastering Bitcoin: Unlocking Digital Cryptocurrencies by Andreas Antonopoulos (1st Edition)
  • Book – Bitcoin and Cryptocurrency Technologies by Princeton University (Draft version)
  • Udemy Ethereum: Decentralized Application Design & Development
  • Diginomics – Ethereum developers course (Created by myself more than a year ago)
  • IBM – Blockchain Essentials for Developers (Includes certification)
  • C4 – Certified Bitcoin Professional.

Princeton University – Bitcoin and Cryptocurrency Technologies

The oldest one still open to the public

Princeton University created one of the first blockchain courses out there, and because of that, they are the first one to be covered in this series. The course is offered since early 2015 (Some of the videos were created in mid-2014). The course is instructed mostly by Arvind Narayanan, Assistant Professor at Princeton computer science department, with few guest appearance by Joseph Bonneau, PostDoc at Princeton, Prof Edward W. Felten, Professor of CS and public affairs at Princeton University.

The course is publicly available on the Coursera platform, and while it might seem as if there is a time limitations, a new class starts every week. Princeton also offers to the public its book “Bitcoin and Cryptocurrency Technologies” (the draft version is free) that follow up the course nicely.

The course is divided into 11 weeks. All weeks are available since day one and there’s nothing that stops you from going through all of the course in one go. Each unit contains few videos, with a simple quiz at the end of each video. You don’t have to solve the quiz to complete that lecture. You’re free to watch the lectures in any order you might want. You don’t have to watch all the videos in order to complete the course.

In order to complete the course, you’ll need to complete three programming assignments.

Course structure

The first three weeks are dealing with the basic aspects of Bitcoin and the blockchain.

It starts by exploring the hash function, and continue to data structure where the hashes are keys. The concept of DAG is hinted in the videos but isn’t entirely covered although Merkle trees are mentioned. A simple Introduction to Asymmetric (Key) cryptography is given but in a very shallow way. ECDSA is also mentioned – ECC is almost completely ignored. RSA isn’t even mentioned. The first week ends with a demonstration of mock cryptocoin (Scroogecoin, we’ll get to that soon).

In the second week, we’re giving a simple introduction to some concepts of decentralization and return to hashes in the context of POW.

The third-week deals with transactions in Bitcoin. The first couple of videos deals with P2PKH and P2SH transactions. The first video also introduce many concepts that are relevant to Bitcoin transactions (UTXO, scripts, inputs, etc.) Scripts are then explored in greater detail. I find the scripts part to be quite well explained. This week ends with an introduction to the concept of blocks and the basics of networking architecture in bitcoin.

All in all, I found the first week to be the weakest in this batch. While the jargon used is aimed at people with background in computer science, many terms were merely glossed over. It made it hard to determine who the course is aimed at – beginners? Individuals with background in computer science? Mathematics? Also, the first assignment (Scroogecoin) requires the students to continue watching all the lectures in week 2 & 3 before being able to solve it. I believe that students taking the course for the first time might try to complete the assignment before moving on, which is another source of frustration.

Week number 4 is kind of stand by itself as it deals mostly with how to store and use bitcoin by looking at wallet types, exchanges and so on. The jargon in this part is more mathematical/technical then week 2-3. Some of the material feels somewhat outdated, but other parts like the way to split and share keys and fees calculation were quite simple and satisfying. However, seeing as most topics during this week might appeal to the more general user, one with little to no background in CS or another related field, I find the return to the more technical jargon being quite counterproductive.

Week four also ends with another assignment. As before, this assignment can’t be solved just by watching the lectures, but I guess that by now most students have already realized it, and won’t postponed watching the rest of the lectures for a time after they’ve managed to solve the assignment.

Week 5 – 11 deals with many different aspects of Bitcoin. Mining, anonymity, legal, and bitcoin as a platform. Each lecture is good by itself as a mean of introducing the students to the basic concepts, but it never goes further than that. In that regard, I see little value in following the structure of the course. It would have been better to just provide the students with a simple glossary/list of topics to cover with links to external resources. Still, some of the lectures were highly enjoyable. In this units I’ve enjoyed the lectures on mining algorithm, which starts to show a simple introduction to the way the hashing functions work, the zerocoin introduction was also useful in providing the foundations for newcomers. Bitcoin as an append-only log especially resonance with my ideas on the blockchain and its use case.

Week 7 (deals mostly with the inner politics of Bitcoin) ends with the last assignment that the students are required to submit. Once again, the assignment has little to do with the lectures themselves.

The assignments:

As mentioned before, the course contains three assignments. All three are in Java and deals with validating a transaction, validating a block, and reaching consensus.

This three task represents a substantial part what is required from a bitcoin node. All three are graded by the use of very impressive test engine that runs multiple test cases on your code and validates the results. The amount of effort placed in creating these assignments is visible and because the course is giving for free, online, and bears the Princeton University name with it, makes these assignments even more valuable.

Which makes it even harder for me to say the following. While I can appreciate the effort putting into these assignments, they feel much more like homework on Java and OOP with a Bitcoin “theme” than a real Bitcoin coding exercises.

The first assignment requires the students to prove some understanding to the way Bitcoin transactions are validated and chained to each other. This is the most Bitcoin-related assignment of them all as it requires some understanding of how bitcoin transactions work. Still, at the end of the day, the real task here is to create an algorithm that will return a specific set of valid transaction and it feels as if a substantial part of the time placed into solving this assignment is spent on recursive loops and working with sets and hash tables rather than working with more aspects that are unique to bitcoin.

Assignment 2-3 were completely off. The second assignment required the students to reach a consensus between nodes in a way that has nothing to do to with how blockchain works. This assignment feels like a pure algorithm task. It’s clear that the creators of the assignment thought carefully about it and worked hard to create fair and interesting tests. After all, more than one algorithm can be implemented to reach a consensus. But at the end, it turns out that the task can be solved by using a ridiculously simple algorithm. I could almost imagine the creators of this assignment slap themselves on the back for managing to create assignment so well though that it can be tested and passed by multiple algorithms with a variety of complexity levels. It is impressive, but have very little to do with the way bitcoin works.

Assignment three starts to focus again on Bitcoin. It requires the students to validate blocks and constructing the blockchain. This assignment is somewhat more realistic and more connected to the way Bitcoin works. But it still requires the students to spend to much time constructing classes, methods, loops and sets – and too little time to really understand how the blockchain is constructed.

The assignments can be submitted as many times as you like, and I highly suggest to try and submit even half-baked codes. The automated grader will run the test engine on your code and might provide you with some interesting insights regarding your codes. For the “less honest” students out there, the code for the test cases (and more) is just a short google away.

The grader engine provides useful information. You can resubmit your files as many time as you like

The course forum

The course has a forum which goes all the way back to 2015. It’s not limited just to the current course timeframe. In this forum, some interesting discussions are taking place, and I highly suggest to join the course if only to read some of the conversations taking place there. These forums also contain a lot of information regarding the assignments which I also suggest looking into.

Graduating and certifications

The course can be completed by submitting the three assignments and receiving a grade of 60 and above. There’s no need to watch all the videos or taking the quiz at the end of each video to complete the course. The assignments can be submitted as many time as you like so that you’re giving the chance to improve your grades.

Upon completion, you’ll receive an email congratulating you for completing the course. You can also verify your identity at Coursera by uploading your current photo plus an office identification (passport or driver license). Once Coursera verifies your identity, you can share your achievement online on Facebook, Linkedin, etc.

The certification issued by Courera and doesn’t bear the Princeton Univesity name. Knowing that the course is free, the assignment has little to do with Bitcoin (and can be easily cheated), this certification carries little to no weight.

You can share your achievement online. No other recognition is provided

Final thoughts

The beginning of the course (weeks 1-3/4) is its the weakest part. Students who want to understand how transactions, keys, hashing, POW, etc. works, will be better to look for it in other places. After that, the course becomes somewhat less technical. This is a welcome change as it allows it to provide a basic introduction to many different aspects of the blockchain.

In this regards, when ignoring the first few weeks, this course might appeal to students who already understand some of the basic concepts of the blockchain and Bitcoin and are just looking for a good source to review as many aspects of it. Each video is no more than 20 minutes. And the list of topics that are covered is quite impressive.

The assignments are wonderful as an exercise in Java and OOP but provide very little extra value for those who are interested in the blockchain. It’s obvious that one can pass the course just by utilizing his/hers experience in OOP while having little understanding of the Bitcoin works.

Completing the course can be achieved in a single day if one chooses, and being able to share your achievement online is a nice bonus, especially when Princeton University is mentioned.

Take this course if:

·       You understand Java and OOP

·       Have some backgrounds in how bitcoin works regarding transactions, keys, blocks, etc. and want to enrich your knowledge of other aspects of it

Time to complete the courseYou can complete the course in one day or eleven weeks. It’s up to you
InteractionThe course forum contains some interesting conversations
Materials

·       Videos mostly

·       The draft of the book Bitcoin and Cryptocurrency Technologies si also available

·       Three programming assignments

CertificationYes, limited. Provided by Coursera for those who complete the three assignments
Bitcore.js keys for zeros and ones.

Bitcore.js keys for zeros and ones.

Bitcore is an excellent JavaScript library that is in use in many Bitcoin-related websites. Using the tools in this library one can easily achieve almost every Bitcoin functionality. Creating key pairs, parsing blocks, creating and signing transactions and more. In this post, I’ll focus on the use of Bitcore when dealing with key pair.

 

The zero

A beginner web developer can create a simple key pair by just using:

bitcore = require("bitcore-lib");
privateKey = new bitcore.PrivateKey // Generate a random private key
privateKey.toString()
>> "23bc740f87bd68729e794eb48b5137150f17109f855a34512c3c5f93d7498291"

The privateKey comes with ample of build in methods, most of which are more than enough for basic Bitcoin implementation in many web applications.

privateKey.toAddress().toString()
>> "1LY75mCpuD3xPfLLFwbWccBNoQQY4VBeQj"
privateKey.toPublicKey().toString()
>> "0379d42509499082436c89a4e1637ed14a27e56174843bddb980ab6f155f82431c"

This codes can be implemented in a manner of minutes. Bitcoin can be incorporated into many web applications in less than a day. However, those of you with a keen eye to Bitcoin protocol probably noticed that the public key comes in its compressed format. The uncompressed public key will be twice as long (64 bytes + 1 byte) and will have 0x04 as their leading byte.

This is, of course, a non-issue for most use cases, but it might be somewhat counterproductive when learning the relations between private keys, public keys, and address.

To truly understand something, there’s no better way than to get your hands dirty. And when we’re referring to something as crucial to Bitcoin (and blockchains in general) as key pairs, no shortcuts should be made.

 

To one.

During my first experimentation with this library, I received the impression that, as good as this library is, it might not be sufficient to teach and learn the underlying mechanism of the Bitcoin blockchain itself.

However, after more careful examination I came to realize that this library provides much more granular tools than what I expected. In fact, it seems that I can use it in conjunction to the developer documentation on Bitcoin addresses.

This is done thanks to one specific method that returns the numeric value of each point in the public key (don’t forget, the uncompressed public key is just a set of x-y coordinates).

publicKey.point.x
publicKey.point.y

This is highly useful method for a number of reasons:

  1. It creates a big number object in JavaScript, thus saves us from the hassle of working with 32+ bytes (A big drawback in JavaScript is the lack of native support for any number that is 32 bytes and above).
  2. Each point is given independently; as a result, the students are more aware of the real meaning of what a public key is really is (in terms of x-y coordinates). It’s also an excellent opportunity to show the students how one coordinate can be dropped to achieve the compressed address.

Another two very useful methods are:

bitcore.crypto.Hash.sha256ripemd160(pubKey);
bitcore.crypto.Hash.sha256sha256(hashedPublicKey);
The first two methods makes the hashing process really easy to execute. The Bitcoin protocol requires double hashing to be executed at two different locations in order to turn the public key to Bitcoin address. In many other languages (such as Python, which I've been using extensively when teaching this subject), there's a need to import and define each hash function. The process is tedious, and at the end of the day, somewhat messy.

By having these two methods, we save almost 40% in code space, and the students can focus on what’s really important in this lesson.

 

And here’s the final result:

// This code represents a possible approach for teaching Keys
// using JavaScript. 
// Pay attention that even while still using a JS library such as bitcore.js 
// The code still follows all the steps defined in the developers documentation.

bitcore = bitcore = require("bitcore-lib");

privKey = bitcore.PrivateKey.fromString("8c5e5b37ebf1e7a274b9dff4910f9a2004868897f7d845802b77a1b245c26bc7");

pubKey = "04" +
        privKey.publicKey.point.x.toBuffer().toString("hex") +
        privKey.publicKey.point.y.toBuffer().toString("hex");

pubKey = new Buffer(pubKey, "hex");

hashedPublicKey = "00" + bitcore.crypto.Hash.sha256ripemd160(pubKey).toString("hex");

hashedPublicKey = new Buffer(hashedPublicKey, "hex");

checkSum = bitcore.crypto.Hash.sha256sha256(hashedPublicKey).toString("hex").slice(0,8);

checkSum = new Buffer(checkSum, "hex");

binAddress = hashedPublicKey.toString("hex") + checkSum.toString("hex");

binAddress = new Buffer(binAddress, "hex");

address = bitcore.encoding.Base58.encode(binAddress)

 

Now compare to what we might see in Python:

Private_key = bytes.fromhex("BF9795D3FCB4E2181B7B536C2247EA0001397C99BA94D4D4DD62801BB151B091")

import ecdsa

signing_key = ecdsa.SigningKey.from_string(Private_key, curve = ecdsa.SECP256k1)

verifying_key = signing_key.get_verifying_key()

public_key = bytes.fromhex("04") + verifying_key.to_string()

import hashlib

sha256_1 = hashlib.sha256(public_key)

ripemd160 = hashlib.new("ripemd160")
ripemd160.update(sha256_1.digest())

hashed_public_key = bytes.fromhex("00") + ripemd160.digest()

checksum_full = hashlib.sha256(hashlib.sha256(hashed_public_key).digest()).digest()

checksum = checksum_full[:4]

bin_addr = hashed_public_key + checksum

import base58

FINALE_BTC_ADDRESS = base58.b58encode(bin_addr)

In this instance, we can see that the code created with bitcore js is much more straightforward and readable than even the simplest Python code.
This is a clear indication that using JavaScript can be a useful tool when teaching the basics of blockchain development.
The libraries can be used when teaching both web developers as well as more protocol oriented students. The code can be presented in highly simplified manner for beginners while still alluding
to its origin and the logic behind it.

Blockchain architecture and JavaScript

Blockchain architecture and JavaScript

Why (and why mot) using JavaScript

Many of my students wishes to create their own blockchain applications. Many of them came from app and web development background, and while they are still highly motivated to learn how the blockchain itself works, their main goal is not purely academic. They want to see a product in action.

When catering for such students, it’s important to adjust the course material to their own background and goals. Mainly it means to provide them with as many JavaScript tools as possible, as it’s the main engine behind many modern websites and apps.

Yet, adjusting the course shouldn’t come at the price of marginalizing the blockchain architecture itself. At the end of the day, the thing that separates such a course from millions of other web/apps development courses is just it – the blockchain.

Bearing this in mind should be a at the heart of every decision taken when creating and conducting such a course. And there’s a very little place for compromises.

This stunt approach led me to prefer teaching the students using more general programming language (mainly, python) while shy away almost completely from using JavaScript, which is fundamentally flawed for teaching protocols such as Bitcoin and the blockchain.

However, when done properly, a substantial part of the Bitcoin protocol can be explained, examined and tested using JavaScrips. And it’s important to develop the proper toolbox of JavaScript codes for that purposes.

 

// This code represents a possible approach for teaching Keys
// using JavaScript. 
// Pay attention that even while still using a JS library such as bitcore.js
// The code still follows all the steps defined in the developers documentation. 

bitcore = require("bitcore-lib");

privKey = bitcore.PrivateKey.fromString("8c5e5b37ebf1e7a274b9dff4910f9a2004868897f7d845802b77a1b245c26bc7");

pubKey = "04" +
        privKey.publicKey.point.x.toBuffer().toString("hex") +
        privKey.publicKey.point.y.toBuffer().toString("hex");

pubKey = new Buffer(pubKey, "hex");

hashedPublicKey = "00" + bitcore.crypto.Hash.sha256ripemd160(pubKey).toString("hex");

hashedPublicKey = new Buffer(hashedPublicKey, "hex");

checkSum = bitcore.crypto.Hash.sha256sha256(hashedPublicKey).toString("hex").slice(0,8);

checkSum = new Buffer(checkSum, "hex");

binAddress = hashedPublicKey.toString("hex") + checkSum.toString("hex");

binAddress = new Buffer(binAddress, "hex");

address = bitcore.encoding.Base58.encode(binAddress);

 

When to use JavaScript

The decision of when, and how to use these codes should be based on 2 main variables.

  1. What the students need to know.

The first one refers to the main reason the students came to the course. The more interested they’re in the blockchain itself, the more they can gain by working with languages such as Python/C++/go etc’. On the other hand, if applications is what they want to create, they can benefit the most by using JavaScrip.

Using JavaScript can be a great experience for them as they’ll both learn how to incorporate many robust libraries into their projects, but (maybe even more important), they’ll understand many of the limitations that surrounds JavaScript and its impact when creating a blockchain app. Limitations such as working on the client side, storing keys, accessing ssl, working with large numbers etc’.

2. What the students already know.

This one is quite simple, what’s the student background. If the student is more competent in JavaScrip, he/she might benefit more from adhering to JavaScript instead of learning another programming language.

 

Can blockchain really be taught using just JavaScript?

I’m still checking it.

I’m trying to migrate as many of the codes that are used for teaching the blockchain itself from Python (most can be found here) to JavaScript.

In the next few posts I hope to provide a more detail accounts on this project, in the hope that it might be more helpful to future students.

 

Mixing environments – Creating working environment for blockchain developers

Mixing environments – Creating working environment for blockchain developers

This article is part of a series of articles depicting my experience with creating and conducting an 8 week long blockchain app development course in Brazil.

 

What tools should be used when teaching blockchain

 

The term blockchain is often misused. Very rarely do people use the term blockchain to describe anything beyond a chain of blocks. A lot of the time when people talk about the blockchain and its application, they basically refer to a somewhat wide variety of technologies, architectures, tools and protocols that, once properly combined and implemented, creates that “blockchain” they are referring to.

When I created the course, it was obvious to me that in order to properly teach the students how to work with “the blockchain”, I’ll first need to spend a lot time dealing with many different technologies and tools. There isn’t just one blockchain IDE or concept to examine; rather there are quite a number of them. Take key pair for example; private and public keys are some of the most crucial (and known) features in many crypto-currencies and blockchains, but they are by no means specific to blockchains. Many people use key pairs off chain. The same holds for many concepts that are highly integrated into the common view about blockchains – Hashing functions, signatures (and keys), scripts and stack architecture, byzantine general problem, bytes codes, merkle trees, DAGs and more.

Each feature in the list above represent another tool/approach/use case/concept that stands by itself but is also crucial to creating what is commonly known as “the blockchain”. This fact posed a great challenge for me when I tried to create the course. It was obvious to me that the course is aimed at people who want to learn how to develop their own blockchain applications and solutions, which meant that it will require the students to get their hands somewhat dirty in codes, command line prompt, and different computational tools.

The challenge here lay in choosing the right tools to work with while remembering that each item on the list should be taught in a manner that is  adequate on the one hand, but without going to a level too deep and insignificant for the course on the other hand. It was also important that there should be a clear difference in the relations between the different and individual items.  I knew I wasn’t hired to teach the students how to program or how to work with different environments. However, making the assumption that they had adequate programming knowledge, enough not to require any introduction to that programing language/ environment/ tools seemed quite optimistic at best, and downright stupid at worst. This is even more so when dealing with a variety of different tools and languages.

I decided to do my best to choose the most user friendly working environments – even at the cost of efficiency and future usability.

Numerous developers have their own working environment. However, I was convinced that every code, example and CLI command/tool should be properly tested and documented in a single uniform environment. The last thing I wanted to do was stand in front of the class while in the background, my code failed to compile. The result of this is that I tried  a lot of different environments while always keeping in mind that the environment to be used should fulfill the following requirements;

  1. It needs to support all the tools I require that my students use.
  2. It shouldn’t affect in anyway the students’ computers, working environments, file systems, paths and/or jeopardizes their computer security in any way.
  3. It should be uniform for all the students.
  4. It should be easy and fast to set and reset whenever needed.
  5. It should be as user friendly as possible.

 

After a few experimentations, I decided to work with the following configurations:

 

  1. Cloud9 level 1 IDE environment with the following installations:
    1. Python-pip.
    2. Python-virtualenv.
    3. Virtual environments for Python 2.7 and 3.5
    4. Ethereum SOLC
    5. Tcpdump (for some reasons, not all c9 workspaces had it installed)
    6. The following pip packages (base58, ecdsa)
Cloud 9 was used for running python files and as a uniform terminal.
  1. Digital ocean Ubuntu 16.041 X64 droplet with the following installations:
    1. Nodejs 6
    2. Meteor Javascript framework version 1.3.4 with web3 and bitcore-lib packages.
    3. The following changes were optional for a few students:
      1. Installing ipfs and running ipfs daemon and adding ipfs-api package to their meteor app. (For those who wished to work with IPFS).
      2. Adding swap file of 4 gb. (For those with memory issues).
  • Use openssh. (More IDE flexibility for advanced users).

 

  1. Solidity browser compiler was mostly used for writing and deploying smart contracts. SOLC (installed on c9) was used by a few students who required some more advanced contracts (mostly when containing libraries).

 

  1. The only 2 components the students were required to install on their own machines were:
    1. Chrome/Chromium with metamask addon.
    2. Wireshark.

 

Cloud9 provided a well-tested and easy to configure working environment that was consistent for all students. It was used mainly to run the Python codes the students created, to compile some Solidity codes (using SOLC), to catch some packets using tcpdump (The tcpdump files were later downloaded and examined using wireshark) and to access digital ocean droplet using ssh.

I was very pleased with this working environment as it was quite robust, highly configurable, not local and easy to reset – Basically it was a great playground to get dirty with, without having to worry about damaging the students’ native environment.

 

Digital ocean droplets were used to provide the students with a uniform platform on which they can create their apps. Meteor is a well-documented JavaScript framework. It was obvious to me that if the students were expected to create applications, they should also have access to some JavaScript tools as both Bitcoin and Ethereum have some very powerful tools for app developers – mainly web3 for Ethereum and Bitcore for Bitcoin.

There’s also another npm package for compiling Solidity (similar to SOLC), but unfortunately, I’ve experienced a lot of compatibility issues with that package and decided to ban the students from using it. IPFS-api is another useful tool for more advanced students who are interested in working with IPFS.

It is important to note that although I did discuss IPFS with some students, I didn’t consider it an important part of the course. First, the system is still in a very early stage. Secondly, the main goal of the course was to teach the students how to develop blockchain applications, and not necessarily decentralized applications (although the two might have a lot in common, they’re not mutually the same) and IPFS just didn’t really fit the slot. Besides, I already had an ample amount of topics to focus on and teach my students (And I must admit; I’m not that much of an expert in this platform myself).

Another point to consider is that in a future course, in the case where there’s no promise to create apps, digital ocean might still be used. In this case, JavaScript libraries can be taught by using clean nodeJS interface.

 

Metamask and solidity browser were wonderful and very easy to use tools. In a manner of minutes, the student had yet another playground to play with Solidity and the Ethereum blockchain.

(It’s important to note that I took some time to make sure ALL of the students were using clean metamask installation WITHOUT any of their real wallets imported to it and only on the Ropsten testnet).

 

One last note about truffle

I also feel compelled to justify a little further my decision to exclude the use of truffle and/or embark (with testrpc) during the course and instead choosing to work with solidity browser compiler. The thing is,  at the time, both truffle and embark had some memory issues that forced me to use another swap file (both when tested on Cloud9 and when tested on digital ocean droplet). In addition to that,  most smart contracts required were easy to deploy from the Solidity web compiler. For specific ad hoc contracts that required the use of a more robust compiler, Ethereum SOLC was used on cloud 9 (SOLC didn’t had any memory issues). I do however recognize that truffle and embark are major tools in the industry and I’m defiantly planning to integrate them into future courses.

Teaching blockchain in Brazil

Teaching blockchain in Brazil

What is it all about

A few months ago, I received an interesting email from a company I didn’t know at the time. What was the content? I was asked to create and conduct a full fledge, 8 weeks long, blockchain development course in Brazil that would involve students from all walks of life.

Blockchain education is something I am passionate about, so this offer immediately struck a chord in me. In the past, I have talked a little on how I got into this field and given some insights on my views about the current ecosystem in terms of education. To sum it up; there’s an existing huge knowledge gap that serves to keep many talented people from properly contributing and utilizing numerous blockchain based solutions.

The deep-seated desire to bridge this gap is a major part of my personal agenda. For that reason, in the past, I created a few videos and tutorials (which owing to the fast paced nature of technology are in sore need of an update, I know) that were aimed at mitigating this gap in knowledge and making blockchain codes somewhat more reachable for developers and tech people who are new to this ecosystem. In addition, I was looking for other ways to bring Bitcoin and Ethereum to the people. In line with this, a creating/conducting course seemed like the logical next step.

In the early chats that transpired between me and the Brazilian company, I was offered the freedom to construct the course as I see fit, without any restrictions.  There was one specific request – that when the duration of the course comes to an end, the student will create their own apps. Considering the proposed length of the course (8 weeks), I believed  it would be quite easily achievable simply by allocating the last 2 weeks of the course to exclusively working with the students on their personal projects.

As we talked about the projects the students might create during the course, we also discussed the lack of needed quality in many of the current blockchain related projects out there.  I gave my impression that it was due to a dearth of understanding of blockchains. Most projects are nothing more than basic apps that have very little to gain from using the blockchain and are either utilizing it for marketing reasons – basically to draw more investors’ money owing largely to a lack of understanding on the investors’ side.

But this is not all; it could also be as a result of a serious flaw in their understanding of how to use the blockchain from the creators’ side. I wanted to ensure the students really have an understanding of what the blockchain is, and equally important, what it isn’t.

My counterpart was impressed by this approach and it became clear to the both of us we were seeing eye-to-eye on what our vision for the course was.

 

Getting ready

Working on the course took a substantial part of my time for the better part of 2-3 months. I basically started creating a course from scratch as there were very limited resources and existing courses to draw from. I created a course structure, list of topics, codes and other teaching aids. The task was a hard one, harder than I initially anticipated. I basically, single handedly tried to create one of the most ambitious courses using nothing but my own means and with literally no support of any kind. But I made it!

Slowly but surely, I managed to create a list of working codes to teach my students. I had a solid course structure, the basic infrastructure for a certification system, few assignments, list of resources, presentations and a lot of notes – all corresponding and completing each other, tested and organized for maximum effect.

 

In Brazil

It was finally the moment of truth. I arrived in Brazil with my course (mostly) neatly organized and prepared. The time for theory was past and it was  time to see it in real action and put my work to the ultimate test.

 

Some curious Brazilian horses that participated my class.

 

The final experience in Brazil was much more challenging than I thought it would be. Some of the difficulties I encountered were related to the technical background and level of the students (the class was very heterogeneous). While some had extensive experience, others had absolutely zero previous experience.  A number of the challenges were related to my own preparation for the said course.

Still, when the course winded to an end, I was highly pleased and proud to see how ALL of the students managed to show great advancement.

I’m very grateful for this experience. I had the opportunity to create a full fledge course, a unique and solid one that provides substantial blockchain education, an element that is sorely lacking outside of very specific programs in selected universities. The challenges I encountered during the course were also a great blessing; my course (and by extension, my whole approach to blockchain education) was tested rapidly and almost every aspect of it was subjected to stress test, and it made it through!

Additionally, this course was a great learning experience for me. As a matter of fact, I’m already working on implementing some of the things I’ve learned in order to provide even better learning resources and courses in the near future.

Now, almost a month after the course, I decided to sit down and articulate my thoughts and experiences. It is my wish to share what I did and what I learned. It is my hope that by doing so, I might be opportune to effect some positive impact on the blockchain education ecosystem and that these articles will be helpful to many.

The articles are currently divided into 7 major categories:

  1. Course structure.
  2. Choosing the working environment.
  3. The codes that were presented and exercised.
  4. Certifications and accreditation.
  5. Teaching aids.
  6. Assignments, homework and evaluation.
  7. Others (students’ background, learning environment, marketing and setting expectations, class size, after class meetings etc.).

These articles will be published in parts in weeks to come.

 

 

blockchain related career

blockchain related career

A pleasant call

A few months ago I was approached by the wonderful guys and girls from the Exosphere Academy with the offer to help them build a new blockahin course. They wanted to create a boot camp for people who are interested in blockchain, its implications and possible implementations. I liked the idea. For some time now I believed that one of the major obstacles for bitcoin, ethereum and other blockchain utilities to achieve their potential lies at the high learning curve new users and developers needs to climb. In this blog, and in other forums, I did my best to try and mitigate this learning curve and I was honored by the opportunity to construct a complete and cohesive course for other developers and enthusiasts.

 

My way.

After I took upon myself the task of creating this course, I was approached by a Brazilian magazine who asked me to describe how I became a blockchain developer. The result was the following article at carreirasolo.org. (The original article is in Portuguese). And right here is an English translation for all of those who’re interested in blockchain related carer.


 

The first time I heard about bitcoin.

Six years ago, I read a short article in a financial magazine about a new type of digital coin called bitcoin. Naturally, the article was completely misleading, which isn’t that surprising. At the time, nobody really understood bitcoin. I’m not even sure the article mentioned the word “blockchain”. But I will always remember this article and how I just skimmed it and completely disregarded the idea as just another scam – thus forfeiting my chance to become a millionaire by downloading a piece of software to my computer. A few years later, in 2012, when I was (slightly) older and wiser, I read another article about bitcoin, this time in a tech magazine. Although it wasn’t completely accurate, the article provided a great service to its audience. The reporter was not only informed about bitcoin, but he also put a lot of effort work into understanding his readers – their technical background and how well informed they were about economics, web architecture, computer science, etc. The result was the first ever bitcoin article I read that dealt with it in terms, analogies and examples that conveyed the actual ideas behind the coin.

After that, I was hooked. The protocol’s promise is one of the most ambitious projects I’ve ever encountered in the digital world. It’s not just another cool app, video game or social network where we can show off pictures to our friends and family. Bitcoin challenges us to literally rethink, remodel and rebuild some of the most central elements of human society – from financial systems, government bodies, bureaucracies, health care, sources of information and much more. All of these can benefit from implementing at least some of the tools presented in the bitcoin protocol. I knew that I wanted to be a part of it.

Baby steps

Initially, I didn’t have much coding experience. At the time I was finalizing my BSC in chemistry and environmental studies. I wrote some very basic Java code to help me in my work, but nothing more than that, so I knew I had to work on my coding skills – sooner rather than later. I decided to study Python for many reasons: it was easy to use, well documented, and already had a (relatively) large selection of bitcoin-related libraries and implementations. I was actually amazed at how well it went. Python turned out to be an extremely easy language to learn, but what was even more encouraging was that while learning the basic concepts of the programing language, I also gained many valuable insights into computer architecture and protocols. All this was very useful when I tried to figure out how Bitcoin worked.

After I felt reasonably competent with Python, I decided to look more into app development. Luckily, at the time, many JavaScript bitcoin-related libraries started to appear. JavaScript was easy enough to learn (especially when all you care about is the bitcoin part, and not the user experience). HTML and CSS were somewhat more confusing, but I soon managed to create some decent-looking interfaces for my bitcoin apps.

During this time, I was constantly reading bitcoin-related news stories, articles, blogs and forums, almost daily. This also provided me with a great feedback loop, as it always helped me focus on the spects of the programming language I was working with (Python and JavaScript).

Introducing ethereum

By the time Vitalik Buterin – who, by the way, was already highly respected In the bitcoin community – introduced his vision for Ethereum, I was already head over heels into Bitcoin and blockchains. I found the approach presented by Vitalik very compelling and promising, and I discovered that by learning how Ethereum works, I could enhance my own understanding of the bitcoin protocol by comparing the similarities and differences between the two.

The big leap

Meanwhile, I started to work on my Master’s degree and took few more advanced courses in algorithms and machine learning to amp up my computer skills. But I wasn’t pleased with the courses. The level at which we learned was high, no doubt about that, but none of the courses offered me any truly useful tools to work with. So I started to study by myself again.

I looked into assembly code, which helped me a lot in my understanding of the Ethereum Virtual Machine (or EVM for short). At this time, I decided to take a leap of faith and turn my passion to blockchains into a full-time career. At first, I started by creating simple videos that explained the basics of bitcoin. Then I showed some implementations for key features using Python. The initial exposure I received was very modest – but it did the job, and almost immediately I started to receive calls from professionals from a variety of industries who asked for my help and advice. Many of them wanted to use the power of bitcoin and the blockchain in their respective industries, and I was thrilled to be at the epicenter of such a great revolution in so many different and unique fields. Many of the people I worked with were genuinely looking for ways to revise their services and models to accommodate the needs of the 21st century. Many have the potential to empower millions (if not billions) of people in developing countries that might finally get direct access to many of the services we take for granted, like banking, medical insurance, pensions, legal frameworks to settle disputes, transparent government … the list goes on and on.

In the meanwhile

Blockchain architecture and computer decentralization is really not the scary thing most people think it is. The protocols are quite well documented, and once you’ve made some mental leaps, it will all start to make sense. You just need to learn it in steps. First, you need to understand what really sets blockchains apart from many other solutions. Then, using some basic coding and IT skills, you can easily familiarize yourself with many of its features and inner workings. The next step is to create some basic code that mimics a few of the more common blockchain-related features. Finally, putting it all together, you can start to create your own blockchain applications and decentralized solutions.

I’m biased. I’m in love with the vision that Bitcoin, Ethereum and blockchain architecture bring to the world. My fingers itch, and my appetite is insatiable. For years I’ve been working diligently to see this revolution spread from the computer geeks and mathematicians to the masses, where it’s really needed. So you’d better take my enthusiasm with a pinch of salt and look for the many blockchain-related projects that will now start to grow – and watch how they’re planning to literally change our world. Then you’ll feel the same way I do. I’m sure of it!

Transaction part one – The misconceptions about the block chain

Transaction part one – The misconceptions about the block chain

Three levels of abstractification.

The are three level to understand the way the Bitcoin block chain works.

The first level
One of the most simple Bitcoin block chain abstractification

In the first level, we got those who just heard about Bitcoin for the first time. People usually thinks that the coins are just associated with the Bitcoin address. Whenever Alice sends Bob coins, she just place a statement (or transaction) in the block chain specifying that the X coins that were associated with Alice address, should now be associated with Bob address. This is of course wrong since this representation contains only 3 thing: the sender address, the receiver address and the amount of coins to be transferred. But this level of abstraction is usually the first thought most people have when first presented with the idea of the block chain.

The second level
Points to previous transacions

In the second level, people starts to understand that each transaction also contains the origin of each coin. So if Alice wants to send 5 BTC to bob, she first needs to show where she got these 5 BTC from, That is, she also needs to provide a way to point to older translations on the block chain, transactions which proves that Alice is indeed the rightful owner of these five BTC. So now what we have starts to look more like a chain. It’s no longer just a table containing the address and the current BTC balance,  rather the block chain needs to contain a list of all previous transactions. This depiction is more accurate, but still it isn’t complete – introducing the third level.

The third level
Points at previous transaction and set the condition for claiming the coins.

In the third level, people learns that a transaction is more then just a simple statement in the block chain. Transaction contains more then just information about the origin of the coins (the input), the address of the receivers (the outputs) and the amount of coins to be transferred. Rather, the transaction also contains a riddle in it, and only by solving this riddle, can the coins be claimed and transferred. So when Alice send 5 BTC to Bob, she doesn’t only pointing at the transaction from which Alice got the coins, it also solves the riddle specified in that transaction to prove that Alice is allowed to claim those coins, but that’s not the end of it, in the transaction that Alice publish on the block chain she’s also inserting another riddle, a riddle that only Bob can answer. So if Bob would like to use this coins in the future, he’ll have to solve that riddle which was provided to him by Alice. And the process goes on and on and on.

I love riddles! Or, how to solve the riddle and prove that I’m allowed to claim the coins?

A quick reminder. Signing messages and key pair.

In the previous post we’ve talked about mathematical trapdoors and hashing functions. We’ve saw how we can use these types of function so sign a message with a private key and how these signed message can be verified with the corresponding public key.

F("My name is Alice", Alice's_private_key) = signed message.

Verify(signed message, "My name is Alice", Alice's_public key) = true.

Change any component in the Verify statement, and we’ll receive false. This way we can easily prove that the message “My name is Alice” was indeed signed by Alice and that no one tempered with that message.

Pay attention! We cannot sign a message with the public key, it simply wont work! the public key can only be used to verify a message, not to sign it!

So, Alice can now publish a message in the block chain that contains the input (origin of the coins) and the output (the address of the receiver). This is the original message, public for anyone to see and check as he or she pleases. Alice can also now specify the following rules in that public message:

  1. This message is the original message.
  2. Within this message I’ve included Bob’s_public_key (In hashed format – We’ll get there in a second).
  3. Take your public key and hash it. If your hashed public key matches the hashed public key from step 2, you may move to the next step.
  4.  Take this original message, and sign it with your own private key.
    F(This message, Bob’s_private_key) = signed message.
  5. The signed message should be verified only against the public key that is specified in this message.
    Verify(signed message, This message, Bob’s_public_key) = true.
  6. If the verification indeed yield true, you may claim the coins in this transaction

 Getting Bob’s public key

When Alice specify Bob as the recipient of a transaction, she does so by inserting Bob’s hashed public key. The hashed public key (as we already saw in the post dealing with keys) can be deducted from Bob’s Bitcoin address.

The public key is hashed and then added to the final Bitcoin address.
The public key is hashed and then added to the final Bitcoin address.

So basically, knowing Bob’s Bitcoin address is equal to knowing Bob’s hashed public key. All we need to do is to convert the address back from base 58, and remember that out of the 25 bytes of the binary address, we need to omit the first byte and the last 4 bytes.

Scrips – the beginning

As we’ve already saw many times in the past, Bitcoin message is no more then a string of bytes that follow a predefined order. That means that in order to specify conditions and rules like the one Alice is using in her transaction message, we also need all parties to agree on a predefined field that will contain the conditions, and we also need to agree on a way to translate the bytes in that field into a set of rules, or instructions. For this purpose, a scripting language was created for Bitcoin, this langue allocate predefined operation to a predefined byte. For example, the byte 0x8b means “plus 1”, the byte 0x8c means “minus 1”, the byte 0xa9 means “hash the public key. First with SHA-256 and then with RIPEMD-160”, and so on. There’re many more of these operators (also called OP-codes) and these OP-codes helps us to specify the rules that needs to be fulfilled. These rules are transparent and are publicly visible on the block chain. Whenever a node verify a transaction, the nodes follows these rules and make sure that they eventually do yield a true statement.

We’ll talk more about these OP-codes and scrips in the next sections, when we’ll also see a real example of valid transaction script.

 

Connection part three – Receiving messages

Connection part three – Receiving messages

In the previous posts, all that we’ve done was to construct and send messages to another node on the network. In this post, we’ll see what happens to incoming messages.

First stop – The ReceiverManager:

class ReceiverManager(Thread):
    def __init__(self, sock):
        Thread.__init__(self)
        self.sendingQueue = Utils.globals.sendingQueue
        self.sock = sock
        self.ping = ""

        self.outfile = open("data_received_from_node.txt", 'w')

    def run(self):
        while True:
            try:

                # get only the header's message
                header = self.sock.recv(24)

                if len(header) <= 0:
                    raise Exception("Node disconnected (received 0bit length message)")

                headerStream = BytesIO(header)
                parsedHeader = HeaderParser(headerStream)

                # get the payload
                payload = self.recvall(parsedHeader.payload_size)
                payloadStream = BytesIO(payload)

                self.manager(parsedHeader, payloadStream)

            except Exception as e:
                print(e)
                break

        print("Exit receiver Thread")

The receivermanager always runs in the background, checking our Thread for any incoming packets. Once it receives a packet, it will immediately cut its first 24 bytes.

header = self.sock.recv(24)

The first 24 bytes are the header. If you remember from this post, every Bitcoin message will starts with header, and the header is always exactly 24 bytes long.

 

The first 24 bytes are the header. The rest is the payload
The first 24 bytes are the header. The rest is the payload.

This header is now parsed as a string of bytes and passed to the HeaderParser class in Bitpy/Network/HeaderParser.py

headerStream = BytesIO(header)
parsedHeader = HeaderParser(headerStream)

 

Second stop – The HeaderParser class:

The HeaderParser class takes the first 24 bytes as a long string of bytes, and then it reads them in the same order that we’ve seen before.

Size (Bytes) Name Data type Description
4 Start string char[4]  The network identifier
12 Command name char[12]  The name of the command.
4 Payload size uint32 Len(payload)
4 Checksum char[4]  SHA256(SHA256(payload))[:4]

First 4 bytes for the Start string (or Magic number), another 12 bytes for Command name, the next 4 bytes are the Payload size and the last 4 bytes are the checksum.

4 bytes for starting string. 12 for command name. 4 for payload size and 4 for checksum
4 bytes for starting string. 12 for command name. 4 for payload size and 4 for checksum
class HeaderParser:
    def __init__(self, header):  # Packets is a stream

        self.magic = read_hexa(header.read(4))
        self.command = header.read(12)
        self.payload_size = read_uint32(header.read(4))
        self.checksum = read_hexa(header.read(4))

        self.header_size = 4 + 12 + 4 + 4

    def to_string(self):
        display = "\n-------------HEADER-------------"
        display += "\nMagic:\t %s" % self.magic
        display += "\nCommand name	:\t %s" % self.command
        display += "\nPayload size	:\t %s" % self.payload_size
        display += "\nChecksum	:\t\t %s" % self.checksum
        display += "\nheader Size:\t\t %s" % self.header_size
        display += "\n"
        return display

We’ve also defined the to_string function which basically makes it easier to print a human readable version of the message header.

You might’ve noticed that currently our code just accept the checksum field from the received message without checking it. This is of course a security flaw in our code. The checksum filed is there to help us verify the authenticity of the message. That is one of the ways we can make sure that no one tempered or changed the message on its way from the sender node to our node. But for the time being we’ll assume that the message is indeed authentic and we’ll accept the checksum as is.

 

Third stop – Back to the ReceiverManager:

Now that we have our header, it’s time to get the payload. The size of the payload was defined in the header of the message. We need to cut that amount of bytes from our incoming packets, just as we cut the first 24 bytes of the header. There’s however one extra step in our code. Instead of using the built in sock.recv function (as we did for the header) we’ve decided to implement our own recevall function. The rational was that since we have no way to predetermine the size of the payload, and since the built in sock.recv can’t handle large packets of unknown size, it would be wiser to break the payload into smaller parts and append them together. This has nothing to do with the Bitcoin protocol, it’s only our way to make sure that the code will properly handle large messages.

def recvall(self, length):
    parts = []

    while length > 0:
        part = self.sock.recv(length)
        if not part:
            raise EOFError('socket closed with %d bytes left in this part'.format(length))

        length -= len(part)
        parts.append(part)

    return b''.join(parts)

So now, after we’ve cut the required amount of bytes that represents the payload of our message, and we have both our header (which was already parsed) and our payload (yet to be parsed), we’ll pass them both to the receivermanager manager function.

 

Forth stop – Manager:

    
def manager(self, parsedHeader, payloadStream):

    command = parsedHeader.command.decode("utf-8")
    message = {"timestamp": time.time(), "command": command, "header": parsedHeader.to_string(), "payload": ""}


    if command.startswith('ping'):
        ping = Ping.DecodePing(payloadStream)

        pong = Pong.EncodePong(ping.nonce)
        packet = PacketCreator(pong)
        self.sendingQueue.put(packet.forge_packet())

        message["payload"] = str(ping.nonce)
        self.display(message)

    elif command.startswith('inv'):
        inv = Inv.DecodeInv(payloadStream)
        message["payload"] = inv.get_decoded_info()
        self.display(message)

    elif command.startswith('addr'):
        addr = Addr.DecodeAddr(payloadStream)
        message["payload"] = addr.get_decoded_info()
        self.display(message)

    elif command.startswith('pong'):
        pong = Pong.DecodePong(payloadStream)
        message["payload"] = pong.get_decoded_info()
        self.display(message)

    elif command.startswith('version'):
        version = Version.DecodeVersion(payloadStream)
        message["payload"] = version.get_decoded_info()
        self.display(message)

The manager function does a very simple thing. It checks the command of the message (the command is part of the header) and then it sends the message payload to be parsed by the corresponding functions. For example. If the manager sees that the command is «pong», it will use the decodepong method in Bitpay/Packets/control_messages/pong.py to extract the desire fields out of it. (You can read more about «pong», «ping» and «verack» messages in this post.).

 

Divergence

We have our pared message, both its header and payload. And now we need to decide what to do with them. For some messages this might be the end of the line. There’s nothing more we can do with them. Some might require us to act. «ping» message should be answered by a «pong» message, transactions should be checked and relayed (We’ll talk about transactions in later posts), «version» messages should be acknowledged by sending back a «verack» message.

A major part of learning the Bitcoin protocol is learning how each and every message should be dealt with. Which fields of information it contains and what is the meaning of this information. We’ve already talked about some of the messages in previous posts  (see here for «ping», «pong» and «verack» messages, and here for «version» message.) and as our project will have more features implemented, so we’ll discuss other type of messages and how to deal with them.