Parallel processing of large graphs from social networks via BSP

Or Erez, Jonathan Karin

Out final project on 'Concurrent and Distributed Programming' course, Ben Gurion University of the Negev.

Table of Contents

• About the Project
  • Built With
• Getting Started
  • Prerequisites
  • Installation
• Usage
• Contact

About The Project

Huge amount of interesting information can be gained by analyzing large-scale data that is derived from social networks.

Social networks are often modeled as graphs, usually known as social graph.

Social graphs of popular social networks (for example: graph that represents all Apple employees on LinkedIn) can contains large amount of vertexes and edges .

Our project offers a distributed soloution for analyzing big graphs, Our project support tree diffrent algorithms:

• Breadth First Search
• Shortest Path Between Two Vertices (Bellman Ford)
• Minimum Spanning Tree

Built With

• Erlang - version 23

Getting Started

Open folder and terminal for the 'master' node, open a folder and terminal for each one of the submasters. in each terimnal write:

git clone https://github.com/JonathanKarin/Graph-BSP
cd Graph-BSP

Submaster:

Do the next stages for every submaster. if you use erlang short name, start erlang: for submaster x :

erl -sname submasterx

if you use erlang long name:

erl -name submasterx@IP -setcookie bsp

compile 'subMaster' and 'worker':

c(subMaster).
c(worker).

start the submaster:

{_,S}=subMaster:start_link().

Master:

if you use erlang short name, start erlang:

erl -sname master

if you use erlang long name:

erl -name master@IP -setcookie bsp

compile 'master' and 'gui':

c(master).
c(gui).

start the program:

gui:start(node()).

Usage

• Choose the file of the graph - txt file that contains all the edges of the graph sorted lexicographically. make sure that this file is exist in all the main folders of all the submasters.
• Choose algorithm.
• Insert the short/long names of all the submasters separated by comma .
• Chhose start/ende vertex.
• Click 'start'
• Wait for answer.

Format:

Contact

Or Erez - orere [at ] post.bgu.ac.il

Jonathan Karin - karinjo [at ] post.bgu.ac.il