Cilk Plus Installation Guide

From SuperTech Wiki

(Difference between revisions)
Jump to: navigation, search
(The PARSEC Cilkification Project (this part is temporarily put here and will be moved.))
 
(28 intermediate revisions not shown)
Line 1: Line 1:
Intel provides a suite of tools for programming Cilk-based programs.  The tool set contains the Intel compiler ICC, the parallelism checker Cilkview, the race condition reporter Cilkscreen, the binary-level instrumentation gadget Pintool, the fine-level instrumentation tool Cilkprof. Here is a guide to install them on a 64-bit Linux.  
Intel provides a suite of tools for programming Cilk-based programs.  The tool set contains the Intel compiler ICC, the parallelism checker Cilkview, the race condition reporter Cilkscreen, the binary-level instrumentation gadget Pintool, the fine-level instrumentation tool Cilkprof. Here is a guide to install them on a 64-bit Linux.  
 +
 +
* '''Online Resources'''
 +
# Open Source Cilk Plus Website: http://www.cilkplus.org/
 +
# Intel Cilk Plus Forum: http://software.intel.com/en-us/forums/intel-cilk-plus
* '''Installation'''
* '''Installation'''
-
# '''Install ICC or GCC:''' You need a license from Intel® in order to install ICC.  A free, open-source version is available in the cilkplus branch of GCC.
+
# '''Install ICC or GCC:'''   A free, open-source version is available in the cilkplus branch of GCC.
-
#* '''Installing ICC:'''  These instructions assume your shell is bash.
+
# '''Installing ICC'''  These instructions assume your shell is bash.
 +
#: ''Prerequisites'':  You need a license from Intel® in order to install ICC.
## Go to http://software.intel.com/en-us/articles/intel-software-evaluation-center/.
## Go to http://software.intel.com/en-us/articles/intel-software-evaluation-center/.
-
## Select "Intel® C++ Composer XE 2011 for Linux" (or any Linux product suite listed as containing "Intel® C++ Composer XE") and download the 64-bit version.
+
## Select the Linux version of "Intel® C++ Composer XE" (or any Linux product suite listed as containing "Intel® C++ Composer XE") and download the 64-bit version.
-
## Unpack the downloaded file and run <code>$ /path/to/unpacked/file/install.sh</code>. To allow all users to access ICC on the system, run this install script as root.
+
## Unpack the downloaded file and run the installation script.
 +
##: <code>$ /path/to/unpacked/file/install.sh</code>.
 +
##: ''Note'': To allow all users to access ICC on the system, run this install script as root.
## Follow the on-screen instructions to install ICC.
## Follow the on-screen instructions to install ICC.
-
## To set up your environment variables to use ICC, run <code>$ source /path/to/intel/bin/compilervars.sh intel64</code>.
+
## Set up your environment variables to use ICC.
-
#* '''Installing GCC:'''  These instructions walk you through checking out and building the cilkplus branch of GCC.  Complete instructions for building GCC from source can be found at http://gcc.gnu.org/install/index.html.  These instructions will install GCC into a custom directory, <code>gcc-cilk</code>, so as not to overwrite the existing version of GCC on your system. As a warning, this process takes a while and uses a couple gigabytes of space.
+
##: <code>$ source /path/to/intel/bin/compilervars.sh intel64</code>.
-
## Check out the latest version of the cilkplus branch of GCC.
+
# '''Installing GCC:'''  These instructions walk you through checking out and building the latest version of GCC, which supports the Cilk keywords.  Complete instructions for building GCC from source can be found at http://gcc.gnu.org/install/index.html.  These instructions will install GCC into a custom directory, <code>gcc</code>, so as not to overwrite the existing version of GCC on your system.
-
##: <code>$ svn checkout svn://gcc.gnu.org/svn/gcc/branches/cilkplus gcc-cilk-src</code>
+
#: ''Warning'': This process takes a while and use a couple gigabytes of space.
-
##: '''Note:''' I encountered problems with my connection timing out when I tried to check out the repository from http://gcc.gnu.org/svn/gcc/branches/cilkplus instead of svn://gcc.gnu.org/svn/gcc/branches/cilkplus.  If you encounter similar problems, make sure you are using the "svn" protocol, instead of the "http" protocol. If you have previously checked out the gcc cilkplus source using "http," you can switch your checkout to use "svn" by running
+
#: ''Prerequisites'': You will need the following installed on your system. (For more information non prerequisites, please check http://gcc.gnu.org/install/prerequisites.html.)
-
##: <code>$ svn switch --relocate http://gcc.gnu.org/svn/gcc/branches/cilkplus svn://gcc.gnu.org/svn/gcc/branches/cilkplus</code>
+
#:: <code>flex</code> or <code>lex</code>
-
##: in the directory containing your checkout. -- TB
+
#:: <code>bison</code> or <code>Yacc</code>
-
## Create two new directories, <code>gcc-cilk-obj</code> and <code>gcc-cilk</code>, in the same directory containing <code>gcc-cilk-src</code>.
+
#:: <code>isl</code>
-
##: <code>$ mkdir gcc-cilk-obj gcc-cilk</code>
+
#:: <code>make</code> or <code>gmake</code>
-
## Install gmp, mpfr, and mpc on your system.
+
#:: <code>gcc</code> or <code>g++</code>, version 4.4 or above
-
## Configure GCC as follows:
+
#:: <code>autogen</code>
-
##: <code>$ cd gcc-cilk-obj</code>
+
#:: <code>autoconf</code>, version 2.64
-
##: <code>$ ../gcc-cilk-src/configure --prefix=../gcc-cilk --enable-languages=c,c++</code>
+
#:: <code>svn</code> or <code>git</code>
-
##: '''Note:''' You may want to add <code>--disable-multilib</code> to the configuration options if multilib is not installed on your system.  Otherwise, you may encounter errors such as:
+
#:: <code>gmp</code>
 +
#:: <code>mpfr</code>
 +
#:: <code>mpc</code>  
 +
## Create some directories for the GCC source, compiled objects, and binaries:
 +
##: <code>$ mkdir /path/to/gcc</code>
 +
##: <code>$ mkdir /path/to/gcc/src</code>
 +
##: <code>$ mkdir /path/to/gcc/obj</code>
 +
## Check out the latest version of GCC.
 +
##: You can check out the repository using Git:
 +
##: <code>$ git clone git://gcc.gnu.org/git/gcc.git gcc/src</code>
 +
## Configure GCC:.
 +
##: <code>$ cd /path/to/gcc/obj</code>
 +
##: <code>$ ../src/configure --prefix=${PWD}/.. --enable-languages=c,c++</code>
 +
##: ''Note:'': You might want to add <code>--disable-multilib</code> to the configuration options if multilib is not installed on your system.  Otherwise, you might encounter errors such as:
##: <code>gnu/stubs-32.h: No such file or directory compilation terminated.</code>
##: <code>gnu/stubs-32.h: No such file or directory compilation terminated.</code>
-
## Compile GCC: <code>$ make</code>
+
## Compile GCC:
-
## Install GCC into gcc-cilk: <code>$ make install</code>
+
##: <code>$ make</code>
-
## The new GCC binary is <code>/path/to/gcc-cilk/bin/gcc</code>.
+
##: ''Note:'': If you have multiple cores on your system, you can speed up this compilation using make's <code>-j<number_of_threads></code> flag.
-
# '''Install Cilkview Cilkscreen:''' Get it from here: http://software.intel.com/en-us/articles/intel-cilk-plus-software-development-kit/The include bin directory in PATH.
+
## Install GCC into <code>/path/to/gcc/</code>:
-
# '''Install Pintool:''' Download pintool from here: http://www.pintool.org/downloads.html. Install.  
+
##: <code>$ make install</code>
-
# '''Install Cilkprof:''' You need Pin revision 43611 in order to run Cilkprof.  These instructions include instructions on downloading and installing this version of Pin for Cilkprof.
+
## The new GCC binary is <code>/path/to/gcc/bin/gcc</code>.
-
## Change to the directory in which you would like to install Cilkprof.
+
## Set up the necessary environment variables.  For example, if you are using bash, execute the following, or add these lines to <code>~/.bashrc</code> to set up these variables automatically when you log in.
-
## Create two new directories, <code>cilkutil</code> and <code>3rdparty</code>.
+
##: <code>$ export PATH=/path/to/gcc/bin:$PATH </code>
 +
##: <code>$ export LD_LIBRARY_PATH=/path/to/gcc/lib:$LD_LIBRARY_PATH</code>
 +
##: <code>$ export LIBRARY_PATH=/path/to/gcc/lib:$LIBRARY_PATH</code>
 +
# '''Install the Cilk Plus tools (cilkview and cilkscreen):''' Download the archive from here: https://www.cilkplus.org/download#block-views-cilk-tools-block-1Unpack the archive and include the <code>bin</code> directory inside the newly-created directory in your <code>$PATH</code>.  For example, if you are using bash, execute the following lines, or add these lines to your <code>~/.bashrc</code>.
 +
##: <code>$ export PATH=path/to/cilktools/bin:$PATH </code>
 +
# '''Install Pintool:''' Download Pin from here: http://www.pintool.org/downloads.html. Unpack the archive.
 +
# '''Install Cilkprof:''' These instructions include instructions on downloading and installing Pin for Cilkprof.  These instructions should work with the latest version of Pin, which is version 62141 at the time these instructions were last updated.  These instructions have not necessarily been tested on newer version of Pin, however.
 +
## Change to the directory <code>cilkprof/install/dir</code> in which you would like to install Cilkprof.
 +
##: <code>$ cd cilkprof/install/dir</code>
 +
## Create a directory <code>3rdparty</code> inside of <code>cilkprof/install/dir</code>.
##: <code>$ mkdir cilkutil 3rdparty</code>
##: <code>$ mkdir cilkutil 3rdparty</code>
-
## Download Pin revision 43611 for Linux from http://www.pintool.org/downloads.html, and place the downloaded archive into <code>3rdparty</code>.
+
## Download Pin for Linux from http://www.pintool.org/downloads.html, and place the downloaded archive into <code>3rdparty</code>.
-
## Download “libzca source to provide access to Cilk metadata” and “cilkprof source example of using Cilk metadata” from http://software.intel.com/en-us/articles/download-intel-cilk-plus-software-development-kit/, and place the downloaded archives into <code>cilkutil</code>.
+
## Download libzca from http://cilkplus.org/download, and place the downloaded archive into the current directory, <code>cilkprof/install/dir</code>.
-
## Run <code>$ cd 3rdparty</code> and then unpack the Pin archive.
+
## Unpack the Pin archive inside of <code>3rdparty</code>.
-
## Rename the folder created from unpacking the archive to <code>pintool</code>.
+
## Create a symbolic link called <code>pintool</code> to the directory created by unpacking the archive.
-
## Run <code>$ cd ../cilkutil</code> and unpack both archives.
+
## If you do not have the necessary permissions to access the <code>intel64</code> subdirectory inside of <code>pintool</code>, run <code>$ chmod -R 755 intel64</code>.
-
## Ensure you have the necessary permissions on the unpacked archives, using <code>chmod</code>.
+
## Unpack the <code>libzca</code> archive inside of <code>cilkprof/install/dir</code>.  Create a symbolic link called <code>cilkutil</code> to the directory created by unpacking this archive.
-
##: <code>$ chmod -R 755 *</code>
+
## Build libzca and cilkprof.
-
## Build libzca as follows.
+
##: <code>$ cd cilkutil</code>
-
##: <code>$ cd zca/src</code>
+
##: <code>$ make</code> (You will need a version of <code>gcc</code> and a version of <code>icc</code> to complete this compilation.)
-
##: <code>$ make OUT=linux64</code>
+
## Download the <code>cilkprof</code> script from http://web.mit.edu/neboat/www/code/cilkprof, and place it in a directory in your <code>PATH</code> environment variable.  For example, place it within the same directory as <code>cilkview</code> and <code>cilkscreen</code>.
-
## Build cilkprof as follows.
+
## Set the CILKUTIL environment variable to point to the <code>cilkutil</code> directory.  For example, if your shell is bash, add the line <code>export CILKUTIL=/path/to/cilkutil</code> to your <code>.bashrc</code>.
-
##: <code>$ cd ../../cilkprof</code>
+
-
##: <code>$ make</code>
+
-
## Download the <code>cilkprof</code> script from http://web.mit.edu/neboat/www/code/cilkprof, and place it in a directory in your PATH environment variable.
+
-
## Set the CILKUTIL environment variable to point to the <code>cilkutil</code> directory.  For example, if your shell is bash, add the following line to your <code>.bashrc</code>.
+
-
##: <code>export CILKUTIL=/path/to/cilkutil</code>
+
-
* '''Brief Usage'''
+
* '''Usage'''
** '''Compile with ICC:''' <code>$ icc cilkprogram.cpp -o cilkprogram</code>
** '''Compile with ICC:''' <code>$ icc cilkprogram.cpp -o cilkprogram</code>
 +
** '''Compile with GCC:''' <code>$ g++ cilkprogram.cpp -o cilkprogram -lcilkrts -ldl</code>
** '''Check parallelism:''' <code>$ cilkview ./cilkprogram</code>
** '''Check parallelism:''' <code>$ cilkview ./cilkprogram</code>
** '''Check race condition:''' <code>$ cilkscreen ./cilkprogram</code>
** '''Check race condition:''' <code>$ cilkscreen ./cilkprogram</code>
-
** '''Pintool:''' TO BE COMPLETED
+
** '''Pintool:''' Please see the Pin manual, e.g. http://software.intel.com/sites/landingpage/pintool/docs/62141/Pin/html/.
** '''Linux perf tool:'''  
** '''Linux perf tool:'''  
*** Show list of counters:  <code>$ perf list</code>         
*** Show list of counters:  <code>$ perf list</code>         
Line 60: Line 85:
*** This command generates two csv output files, <code>cilkprogram.bb.csv</code> and <code>cilkprogram.cc.csv</code>, containing work and span profiling data for your Cilk program.  At this time, the data in these files are most easily perused using your favorite spreadsheet program.  For more information on configuring the output of Cilkprof, run <code>$ cilkprof -h</code>.
*** This command generates two csv output files, <code>cilkprogram.bb.csv</code> and <code>cilkprogram.cc.csv</code>, containing work and span profiling data for your Cilk program.  At this time, the data in these files are most easily perused using your favorite spreadsheet program.  For more information on configuring the output of Cilkprof, run <code>$ cilkprof -h</code>.
-
 
+
* '''Alternative implementations of Cilk'''
-
 
+
** '''LLVM/Clang with Cilk Plus:''' Please see http://cilkplus.github.io/ for installation details.
-
= <br>The PARSEC Cilkification Project (this part is temporarily put here and will be moved.) =
+
-
We are interested in bringing Cilk parallelization to PARSEC, an industry-strength benchmark suite for emerging projects. The suite contains 13 projects: x264, a H.264 standard-compliant video stream encoder; facesim, a realistic human facial muscle simulation module; raytrace, a ray-tracing engine; vips: an large-image-oritneded image processing engine; freqmine, a data mining engine that can operate online; fluidanimate, a fast fluid animation module in 3D (faster than Joe Stam's algorithm); streamcluster, a clustering algorithm that can operate online; bodytrack, an algorithm to reconstruct the human pose in different video sequences providing a prior knowledge of the skeleton; canneal, a program to improve the circuit routing planning of a particular circuit with the help of the simulated annealing algorithm; swaption, an option pricing simulation engine; blackscholes, a stock market analysis algorithm; dedup, a compression engine that can operate online over a continuous data stream; ferret, an image retrieval engine. We are also interested in finding common patterns in these programs, and make them standard features of the Cilk language.
+
-
 
+
-
This is a brief description of our understanding and progress on each projects. If you have questions, please email to jzz@mit.edu for answers.
+
-
 
+
-
x264: x264 simulates the H.264 protocol. It concerns video. Many image frames comes together, they are played continuously, this is called video. A video can be separated into images, and an image can separated into blocks. A block can easily find a similar block in the video stream. 90%+ of the time in the x264 module is spent in finding similar blocks for blocks. The one line pseudocode looks likefor each frame f { for each block b in f { find a way to encode b;  write encoded b to disk; }}.  The "find a way to encode b" is to find similar blocks of b. There are several ways to encode a block b: 1. as JEPG; 2. as an increment over a block from the same video frame; 3. as an increment over a block from a previous video frame; 4. as an increment over a linear combination of two blocks from potentially different previous video frames. The simplest way to parallelize is to parallelize the inner loop, although that might involves removing the second type of encoding. This is the parallelization we are trying to implement. [Project status:] code mostly read, parallelization strategy made, no implementation.
+

Latest revision as of 19:16, 5 March 2015

Intel provides a suite of tools for programming Cilk-based programs. The tool set contains the Intel compiler ICC, the parallelism checker Cilkview, the race condition reporter Cilkscreen, the binary-level instrumentation gadget Pintool, the fine-level instrumentation tool Cilkprof. Here is a guide to install them on a 64-bit Linux.

  • Online Resources
  1. Open Source Cilk Plus Website: http://www.cilkplus.org/
  2. Intel Cilk Plus Forum: http://software.intel.com/en-us/forums/intel-cilk-plus
  • Installation
  1. Install ICC or GCC: A free, open-source version is available in the cilkplus branch of GCC.
  2. Installing ICC These instructions assume your shell is bash.
    Prerequisites: You need a license from Intel® in order to install ICC.
    1. Go to http://software.intel.com/en-us/articles/intel-software-evaluation-center/.
    2. Select the Linux version of "Intel® C++ Composer XE" (or any Linux product suite listed as containing "Intel® C++ Composer XE") and download the 64-bit version.
    3. Unpack the downloaded file and run the installation script.
      $ /path/to/unpacked/file/install.sh.
      Note: To allow all users to access ICC on the system, run this install script as root.
    4. Follow the on-screen instructions to install ICC.
    5. Set up your environment variables to use ICC.
      $ source /path/to/intel/bin/compilervars.sh intel64.
  3. Installing GCC: These instructions walk you through checking out and building the latest version of GCC, which supports the Cilk keywords. Complete instructions for building GCC from source can be found at http://gcc.gnu.org/install/index.html. These instructions will install GCC into a custom directory, gcc, so as not to overwrite the existing version of GCC on your system.
    Warning: This process takes a while and use a couple gigabytes of space.
    Prerequisites: You will need the following installed on your system. (For more information non prerequisites, please check http://gcc.gnu.org/install/prerequisites.html.)
    flex or lex
    bison or Yacc
    isl
    make or gmake
    gcc or g++, version 4.4 or above
    autogen
    autoconf, version 2.64
    svn or git
    gmp
    mpfr
    mpc
    1. Create some directories for the GCC source, compiled objects, and binaries:
      $ mkdir /path/to/gcc
      $ mkdir /path/to/gcc/src
      $ mkdir /path/to/gcc/obj
    2. Check out the latest version of GCC.
      You can check out the repository using Git:
      $ git clone git://gcc.gnu.org/git/gcc.git gcc/src
    3. Configure GCC:.
      $ cd /path/to/gcc/obj
      $ ../src/configure --prefix=${PWD}/.. --enable-languages=c,c++
      Note:: You might want to add --disable-multilib to the configuration options if multilib is not installed on your system. Otherwise, you might encounter errors such as:
      gnu/stubs-32.h: No such file or directory compilation terminated.
    4. Compile GCC:
      $ make
      Note:: If you have multiple cores on your system, you can speed up this compilation using make's -j<number_of_threads> flag.
    5. Install GCC into /path/to/gcc/:
      $ make install
    6. The new GCC binary is /path/to/gcc/bin/gcc.
    7. Set up the necessary environment variables. For example, if you are using bash, execute the following, or add these lines to ~/.bashrc to set up these variables automatically when you log in.
      $ export PATH=/path/to/gcc/bin:$PATH
      $ export LD_LIBRARY_PATH=/path/to/gcc/lib:$LD_LIBRARY_PATH
      $ export LIBRARY_PATH=/path/to/gcc/lib:$LIBRARY_PATH
  4. Install the Cilk Plus tools (cilkview and cilkscreen): Download the archive from here: https://www.cilkplus.org/download#block-views-cilk-tools-block-1. Unpack the archive and include the bin directory inside the newly-created directory in your $PATH. For example, if you are using bash, execute the following lines, or add these lines to your ~/.bashrc.
    1. $ export PATH=path/to/cilktools/bin:$PATH
  5. Install Pintool: Download Pin from here: http://www.pintool.org/downloads.html. Unpack the archive.
  6. Install Cilkprof: These instructions include instructions on downloading and installing Pin for Cilkprof. These instructions should work with the latest version of Pin, which is version 62141 at the time these instructions were last updated. These instructions have not necessarily been tested on newer version of Pin, however.
    1. Change to the directory cilkprof/install/dir in which you would like to install Cilkprof.
      $ cd cilkprof/install/dir
    2. Create a directory 3rdparty inside of cilkprof/install/dir.
      $ mkdir cilkutil 3rdparty
    3. Download Pin for Linux from http://www.pintool.org/downloads.html, and place the downloaded archive into 3rdparty.
    4. Download libzca from http://cilkplus.org/download, and place the downloaded archive into the current directory, cilkprof/install/dir.
    5. Unpack the Pin archive inside of 3rdparty.
    6. Create a symbolic link called pintool to the directory created by unpacking the archive.
    7. If you do not have the necessary permissions to access the intel64 subdirectory inside of pintool, run $ chmod -R 755 intel64.
    8. Unpack the libzca archive inside of cilkprof/install/dir. Create a symbolic link called cilkutil to the directory created by unpacking this archive.
    9. Build libzca and cilkprof.
      $ cd cilkutil
      $ make (You will need a version of gcc and a version of icc to complete this compilation.)
    10. Download the cilkprof script from http://web.mit.edu/neboat/www/code/cilkprof, and place it in a directory in your PATH environment variable. For example, place it within the same directory as cilkview and cilkscreen.
    11. Set the CILKUTIL environment variable to point to the cilkutil directory. For example, if your shell is bash, add the line export CILKUTIL=/path/to/cilkutil to your .bashrc.
  • Usage
    • Compile with ICC: $ icc cilkprogram.cpp -o cilkprogram
    • Compile with GCC: $ g++ cilkprogram.cpp -o cilkprogram -lcilkrts -ldl
    • Check parallelism: $ cilkview ./cilkprogram
    • Check race condition: $ cilkscreen ./cilkprogram
    • Pintool: Please see the Pin manual, e.g. http://software.intel.com/sites/landingpage/pintool/docs/62141/Pin/html/.
    • Linux perf tool:
      • Show list of counters: $ perf list
      • check L1 data load miss: $ perf stat -c L1-dcache-load-misses ./cilkprogram
    • Cilkprof: $ cilkprof ./cilkprogram
      • This command generates two csv output files, cilkprogram.bb.csv and cilkprogram.cc.csv, containing work and span profiling data for your Cilk program. At this time, the data in these files are most easily perused using your favorite spreadsheet program. For more information on configuring the output of Cilkprof, run $ cilkprof -h.
Personal tools
Members Only
Off Topic