Saturday, April 17, 2010

1. Building P2P Simulators with OverSim - Where to Begin

This could be a series of blog posts about extending or developing your own OverSim applications & overlay networks. OverSim has a minimal tutorial on writing your own application & overlay network; however, it doesn't show the big picture. So, I'm wasting lots of time playing with code & trying to understand the rest. Good thing is, I like it more & more as I understand. You need to change/develop only a few things, but finding out which ones is a hell of a task. I hope this will not only make my life easy but also will be useful to new comers. Here's what you need to do:
  1. You need some background on OMNeT++
    • OverSim extend OMNeT++. But sometime it has its own way of doing things (to make your task even simple) so understand the differences.
    • Develop several OMNeT++ simulators. TicToc is a good one to start with. Extend it as you imagine.
  2. Read Towards a Common API for Structured Peer-to-Peer Overlays, which is the basis for OvseSim's API. Here's a summary of the paper:
    • Motivate multi-tier application design & define role of layers. OvseSim's overlay layer belongs to Tier 0 while its Application layer captures Tier 1 & 2
    • A key is 160-bits long.
    • root is the node that is responsible for storing a given key
    • nodeHandler encapsulates an overlay node with an ID (within the same key space), IP address, & optionally a port no.
    • msg (a message) contains application data which could be of arbitrary length & format.
    • route(key k, msg m, nodeHandler hint) - Route message m to root of key k. hint (optional) could be use to indicate first hop node in routing. hint is useful when sending messages directly to the destination (given that you already know it).
    • forward(key k, msg m, nodeHandle nextHop) - An up-call to application layer that provides flexibility while routing. Can be call in each intermediate & the destination nodes. Useful while implementing key caching, where application could check m to see whether it already know the destination k. If so, it can directly reply to sender & stop the propagation of m by setting nextHop=NULL. Also useful while implementing malicious nodes that drop messages.
    • deliver(key k, msg m) - Up-call to application layer of the node that is the root of k. Called when m is delivered to destination.
    • localLookup(key k, int n) - A local-call to overlay to find out list of n nodes that can be used as candidates to reach key k. Useful in protocols like Kademlia.
    • neighborSet(int n) - A local-call to find out list of n neighboring nodes in the key space.
    • replicaSet(key k) - Return an ordered set of nodeHandlers on which replicas of object with key k can be stored. Useful while replicating objects. Also useful as nodes join/leave.
    • update(nodeHandler n, bool join) - An up-call to indicate application layer that node n has join/leave the overlay. Useful when aggregating/splitting key ranges as nodes leave/join.
    • range(nodeHandler n) - Get the range of keys that node n is responsible for. Useful in aggregating/splitting key ranges as nodes leave/join.
  3. Read OverSim documentation. Be aware, it may not be up to date.
    1. OverSimInstall - How to install OMNeT++, INET, & OverSim.
    2. OverSimUsage - How to simulate existing examples & plot data.
    3. OverSimDevelop - How to add a new application & overlay protocol.
    4. OverSimAPI - Module & API documentation, can't live without this.
    5. Write your own code, extend existing code as you imagine. You are guaranteed to spend days here before building your real simulator.
    6. Optional
      1. OverSim: A Flexible Overlay Network Simulation Framework
      2. OverSimChurn - How to use different churn generators.
      3. OverSimKbrRouting - Description of different KBR routing types

28 comments:

Anonymous said...

I would like to exchange links with your site www.blogger.com
Is this possible?

Dilum Bandara said...

I'm not sure what you mean. Do you want to link to my blog?

Anonymous said...

hello.Can we install oversim in windows??

Dilum Bandara said...

Yes. Please see the installation instructions on OvserSim site. I personally recomend Linux installation.

Anonymous said...

In the installation guide for oversim, under windows it is stated that :
•For Windows: Everything you need is already included in the OMNeT++ 4.0 package.

I have already installed OMNeT++ 4, but in the project explorer i cannot see the oversim project folder.

I want to use oversim to build a structured P2P network. Will there be a problem using windows 7 for this??

Dilum Bandara said...

What it means is "everything that you need to install OMNeT++ on Windows is included in installation" (no need to install additional packages like in Linux).
You need to follow the of the remaining steps to install INET & OverSim. You should be able to install on Windows 7.

Anonymous said...

Hello..
I am trying to build an application that will use the pastry protocol to distribute files/jobs in the P2P environment.

I'm a new Oversim user and far I can manage to run the pastry simulation and change configuration in Oversim.

Any help to get me started will be deeply appreciated.

Dilum Bandara said...

Do you want to use Pastry as it is or modify its behavior? If it's former, go though application & other layers (tier1, tier2, tier3). Because you want to index files you may want to look at DHT & DHTTestApp codes. If it's later you can still start with DHT & DHTTestApp then go through Pastry code as well. You may want to write summary & draw diagrams as you understand things. This will be few days works unless you are an experienced C++ programmer.

Anonymous said...

I want to modify how pastry handles fault tolerance and then test performance of my modification.
-Possible things i want to change include nodes having the ability to send "message of desire to leave the network" and making pastry to handle such messages.

for my test application, i want the bootstrap node to be able to send job messages to the nodes in pastry network. at the end of the simulation time, i want to check how many of the jobs sent by the bootstrap message wer returned.

i'm not really an experienced c++ programmer and yes i have started drawing diagrams of the class and the relationships.

Anonymous said...

Hello,
I has to implement bittorrent with oversim, but I am lost
I managed to install oversim, I even well read all the examples
I downloaded an old version of bittorrent
And I want to start from 0

If you can guide me a little more please
Your aricle pleased me but I need more help

Thank you very much

Dilum Bandara said...

See http://groups.google.com/group/oversim/browse_thread/thread/32eab98605294567/e79222e028ac3074?lnk=gst&q=BitTorrent#e79222e028ac3074
You may be able to modify it rather than implementing everything yourself.

Anonymous said...

< You may be able to modify it rather than implementing everything yourself. >
Thanks for answering

Anonymous said...

Hello
I've installed "Omnet++ 4.1" and "INET-OverSim-20101019" in Win XP properly, but while installing "OverSim-20101103" some problems happened.
I've followed installation guide step by step.
at first I used mingwenv to install, it worked for omnet++ and INet but didn't work for Oversim, Then I tried Eclipse on Omnet++ and got nothing but this error as I had seen it in minwenv:
"underlay/simpleunderlay/SimpleUnderlayConfigurator.cc: In member function 'uint32_t SimpleUnderlayConfigurator::parseCoordFile(const char*)':
underlay/simpleunderlay/SimpleUnderlayConfigurator.cc:306:18: error: 'malloc_trim' was not declared in this scope
make: *** [../out/gcc-debug/src/underlay/simpleunderlay/SimpleUnderlayConfigurator.o] Error 1"

What should I do?

Dilum Bandara said...

This is a bug in new version. See http://groups.google.com/group/oversim/browse_thread/thread/3a68fcb018c1f45/8f7b2d094b55bac0?lnk=gst&q=malloc_trim#8f7b2d094b55bac0

Wathsala Vithanage said...

Hi,
Is there a CAN implementation for OverSim?

Dilum Bandara said...

No, as far as I know...
You can try to build your own

Anonymous said...

hi, new to the site, thanks.

Bhushan Kakulte said...

Can u tell how to take readings in oversim? Lets take example that I want to built graph for chord number of nodes vs no of message exchange....

Harisankar H said...

Oversim claims that it is very easy to port a simulation code to real deployment. Have you tried and verified it before ? What is your opinion on it ?

Thanks.

Dilum Bandara said...

I haven't tried. Because it's object oriented you should be able to reuse many methods/functions particularly related to Application layer & above. You would need to good understand about existing framework to determine what can be used & what can't.

Anonymous said...

i am making a project in oversim of having a replication system, can u guide me from where to start with?

Anonymous said...

Can u tell how to take readings in oversim? Lets take example that I want to built graph for DHT number of nodes vs no of message exchange....

Dilum Bandara said...
This comment has been removed by the author.
Dilum Bandara said...

For a start:
Follow steps given at http://dilumb.blogspot.com/2010/04/building-your-own-p2p-simulator-with.html
When it comes to implementation, you are pretty much alone.

To collect stat:
Use a variable(s) to collect no of messages at each node. You may need multiple variables as there are different types of messages. Then dump them at end of the simulation using something similar to following

RECORD_STATS(globalStatistics->addStdDev("DHT: Put messages", numPutMessages));

This can be added to finish() function. See some examples on OverSim (like DHTTestApp). Finally, you may need to write a small script to count messages from all the nodes

belzeebob said...

hello brother, im so sorry if my question might be redundant. i have
looked all over the web and still can't find solutions. Im not one of
the lucky ones who were able to install omnet 4.1 and oversim on win7.
After stumbling upon a post, i was able to build inetoversim through
the omnet ide by importing it then compiling it with dll as the
target.
Now still the same problem, im having trouble with oversim 20101103
itself. What is the right way to build it within the omnet IDE?
I tried a lot of times using the compiler that comes with Omnet(mingw) but i keep getting errors which looks for dll's (i add the a .dll and it
will look for another one) until i just get tired.
Now, i tried to build it within Omnet so i was able to build
inet, and im getting desperate about how to build oversim and use it.
I didnt touch anything in the code so i don't think there's some kind
of coding error.
Anyway, here are the errors it generated:
make: *** [../out/gcc-debug/src/libOverSim.dll] Error 1 OverSim line
0 C/C++ Problem
undefined reference to `_imp____gmpn_add_n' OverlayKey.cc /OverSim/src/
common line 249 C/C++ Problem
undefined reference to `_imp____gmpn_lshift' OverlayKey.cc /OverSim/
src/common line 386 C/C++ Problem
undefined reference to `_imp____gmpn_rshift' OverlayKey.cc /OverSim/
src/common line 365 C/C++ Problem
undefined reference to `_imp____gmpn_set_str' OverlayKey.cc /OverSim/
src/common line 115 C/C++ Problem
undefined reference to `_imp____gmpn_set_str' OverlayKey.cc /OverSim/
src/common line 662 C/C++ Problem
undefined reference to `_imp____gmpn_sub_n' OverlayKey.cc /OverSim/src/
common line 258 C/C++ Problem

Is there something wrong with omnet's compiler?
PLease help.

belzeebob said...

hello brother, im so sorry if my question might be redundant. i have
looked all over the web and still can't find solutions. Im not one of
the lucky ones who were able to install omnet 4.1 and oversim on win7.
After stumbling upon a post, i was able to build inetoversim through
the omnet ide by importing it then compiling it with dll as the
target.
Now still the same problem, im having trouble with oversim 20101103
itself. What is the right way to build it within the omnet IDE?
I tried a lot of times using the compiler that comes with Omnet(mingw) but i keep getting errors which looks for dll's (i add the a .dll and it
will look for another one) until i just get tired.
Now, i tried to build it within Omnet so i was able to build
inet, and im getting desperate about how to build oversim and use it.
I didnt touch anything in the code so i don't think there's some kind
of coding error.
Anyway, here are the errors it generated:
make: *** [../out/gcc-debug/src/libOverSim.dll] Error 1 OverSim line
0 C/C++ Problem
undefined reference to `_imp____gmpn_add_n' OverlayKey.cc /OverSim/src/
common line 249 C/C++ Problem
undefined reference to `_imp____gmpn_lshift' OverlayKey.cc /OverSim/
src/common line 386 C/C++ Problem
undefined reference to `_imp____gmpn_rshift' OverlayKey.cc /OverSim/
src/common line 365 C/C++ Problem
undefined reference to `_imp____gmpn_set_str' OverlayKey.cc /OverSim/
src/common line 115 C/C++ Problem
undefined reference to `_imp____gmpn_set_str' OverlayKey.cc /OverSim/
src/common line 662 C/C++ Problem
undefined reference to `_imp____gmpn_sub_n' OverlayKey.cc /OverSim/src/
common line 258 C/C++ Problem

Is there something wrong with omnet's compiler?
PLease help.

Dilum Bandara said...

I'm not familiar with OverSim on Windows. Only thing I can point to is installation steps on https://sites.google.com/site/oversimdemystified/

Naveen Kumar said...

i want to impliment 2 tier dht using oversim please guide me