session3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: Integrating GraphSpace into network analysis projects - Session 3\n",
    "\n",
    "**Presenters**: Aditya Bharadwaj, Jeff Law and T. M. Murali\n",
    "\n",
    "Helpful links:\n",
    "- [2017-ICSB-graphspace-tutorial GitHub repo](https://github.com/Murali-group/2017-ICSB-graphspace-tutorial)\n",
    "- [2017 ICSB GraphSpace tutorial Session 2 notebook](session2.ipynb)\n",
    "- [GraphSpace](http://graphspace.org)\n",
    "- [GraphSpace documentation](http://manual.graphspace.org)\n",
    "- [GraphSpace Python Client’s documentation](http://manual.graphspace.org/projects/graphspace-python)\n",
    "- [Cytoscape JS styles](http://js.cytoscape.org/#style)\n",
    "- [Jupyter](http://jupyter.readthedocs.io/en/latest/install.html) or [IPython](https://ipython.org/install.html)\n",
    "\n",
    "\n",
    "About Me:\n",
    "  - Jeffrey Law\n",
    "  - PhD Student, Genetics, Bioinformatics and Computational Biology at Virginia Tech\n",
    "  - Email: jeffl@vt.edu\n",
    "\n",
    "## Agenda\n",
    "1. Overview\n",
    "2. PathLinker Analysis\n",
    "  - Protein-protein interaction network\n",
    "  - k shortest paths\n",
    "3. Post results to GraphSpace\n",
    "  - Read in PathLinker results\n",
    "  - Add style, popups, path rank filter and more attributes\n",
    "4. Reconstruction of the WNT Signaling Pathway\n",
    "5. Post your own networks!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "In this session we will learn how to post more complex graphs to GraphSpace programatically. \n",
    "\n",
    "One of the main goals of this tutorial is to help you share and interact with your own networks online at GraphSpace (www.graphspace.org).  We suggest you bring your network(s) in a tab-delimited file of edges and have an idea of some of the visual properties (colors, edge style, node shapes) you want to use and we'll help you make it interactable and shareable online in this last session.\n",
    "\n",
    "I have an example workflow where we will be repeating the analysis done by Ritz et al. [1] to reconstruct the WNT signaling pathway using PathLinker. In their analysis, they reconstructed signaling pathways using the set receptors and transcription factors (TFs) of a pathway as sources and targets for PathLinker. I have provided the receptors and TFs of the WNT pathway as well as an updated version of the human protein-protein interaction network (background interactome) for us to repeat their reconstruction of the WNT pathway. The figure below shows an overview of their analysis.\n",
    "\n",
    "![netpath-wnt](images/method-overview.png)\n",
    "  1. Ritz et al., [Pathways on demand: automated reconstruction of human signaling networks](https://www.nature.com/articles/npjsba20162). npj: Systems Biology and Applications, 2016\n",
    "\n",
    "In the example workflow, you will learn how to take a list of edges create a high-quality graph sharable online.\n",
    "More specifically you'll learn to programatically:\n",
    "1. Set shapes, colors, and styles to nodes and edges based on chosen node and edge properties\n",
    "2. Add filters to your graph to focus on different pieces or sections of the graph\n",
    "3. Add important node and edge information to popups that link to additional information\n",
    "\n",
    "After posting those results to GraphSpace you should have a pretty good idea how to get your own networks onto GraphSpace.\n",
    "\n",
    "### What we're posting today:\n",
    "![netpath-wnt](images/NetPath-WNT-PathLinker-k100.png)\n",
    "[WNT Pathway PathLinker Reconstruction k100](http://graphspace.org/graphs/22587?user_layout=1411)\n",
    "Blue nodes and edges are in the NetPath WNT pathway, purple nodes and edges are in the KEGG WNT pathway.\n",
    "Here is what the popups will look like:\n",
    "![popup](images/popup.png)\n",
    "\n",
    "### Example Networks on GraphSpace\n",
    "Below I have a little showcase of public graphs I and other users have created.\n",
    "![kegg-wnt](images/KEGG-Wnt-signaling-pathway-with-ranks.png)\n",
    "[KEGG Wnt signaling pathway](http://graphspace.org/graphs/22527)\n",
    "![5-fu](images/5-Fluorouracil-Case-Study.png)\n",
    "[5-Fluorouracil Case Study graph](http://graphspace.org/graphs/22527)\n",
    "![ACME](images/ACNE_Support=1.0Confidence=30.0.png)\n",
    "[ACNE_(Support=1.0%, Confidence=30.0%)](http://graphspace.org/graphs/22247)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Packages to install\n",
    "We'll be using the following packages in this tutorial.\n",
    "- [graphspace_python](https://github.com/adbharadwaj/graphspace-python). This tutorial was made using v0.8.2\n",
    "- [pandas](https://pandas.pydata.org/pandas-docs/stable/10min.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you're using anaconda, you can install pip using the following command\n",
    "!conda install -y pip\n",
    "# Install these packages using pip.   \n",
    "!pip install graphspace_python==0.8.2 pandas\n",
    "# If you're not using anaconda and you're running python3, then use pip3 instead.\n",
    "# --user tells pip to install locally without needing root priviledges\n",
    "#!pip3 install --user graphspace_python==0.8.2 pandas\n",
    "# Now download PathLinker\n",
    "!conda install -y git\n",
    "!git clone \"https://github.com/Murali-group/PathLinker\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Introduction to PathLinker\n",
    "\n",
    "PathLinker computes the k shortest paths from a set of sources to a set of targets in a background graph/network. PathLinker uses [Yen's Algorithm](https://en.wikipedia.org/wiki/Yen%27s_algorithm) to find the k shortest paths.\n",
    "\n",
    "PathLinker inputs: \n",
    "  - (Weighted) Background interaction network\n",
    "  - Set of sources and targets\n",
    "  - k # of shortest paths to compute\n",
    "  \n",
    "PathLinker output:\n",
    "  - The k shortest paths sorted by path length/score\n",
    "  \n",
    "![linker-overview](images/linker-overview-nih-proposal.svg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run PathLinker\n",
    "### Protein-protein interaction network\n",
    "Protein-protein interactions (PPIs) are available from multiple databases for many species. \n",
    "We work with Human and Yeast \n",
    "\n",
    "Sources: BioGrid, DIP, InnateDB, IntAct, KEGG, MatrixDB, MINT, NetPath, PhosphositePlus, Reactome, SPIKE'\n",
    "  - 35K regulatory (directed) PPIs\n",
    "  - 200K physical (undirected) PPIs\n",
    "  \n",
    "Edges are weighted using an evidence-based probabilistic approach.\n",
    "I also provideda file containing the evidence for each interaction including interaction type, experimental detection method, pubmed IDs and source database for each interaction and a mapping file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can use pandas to look at the files in a nice table form\n",
    "import pandas as pd\n",
    "df = pd.read_csv(\"data/2017-06-16-human-ppi-weighted-cap0_75.tsv\", sep='\\t') \n",
    "print(\"\\n\\tdata/2017-06-16-human-ppi-weighted-cap0_75.tsv\")\n",
    "print(df.head(3))\n",
    "df = pd.read_csv(\"data/2017_06-human-interactome-evidence.tsv\", sep='\\t') \n",
    "print(\"\\n\\tdata/2017_06-human-interactome-evidence.tsv\")\n",
    "print(df.head(3))\n",
    "df = pd.read_csv('data/2017_06-human-interactome-mapping.tsv', sep='\\t', header=None, names=[\"uniprotID\", \"gene_name\"])\n",
    "print(\"\\n\\tdata/2017_06-human-interactome-mapping.tsv\")\n",
    "print(df.head(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sources and Targets\n",
    "Using the WNT pathway receptors as sources and the transcription facotrs (TFs) as targets for PathLinker, we can build a reconstruction of the WNT pathway."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"data/Wnt-sources-targets.tsv\", sep='\\t') \n",
    "print(\"\\n\\tdata/Wnt-sources-targets.tsv\")\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running PathLinker\n",
    "Now that we have all of the inputs set, we can run PathLinker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python PathLinker/PathLinker.py --help"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python PathLinker/PathLinker.py \\\n",
    "    data/2017-06-16-human-ppi-weighted-cap0_75.tsv \\\n",
    "    data/Wnt-sources-targets.tsv \\\n",
    "    -k 100 \\\n",
    "    -o output/Wnt- \\\n",
    "    --write-paths "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!head output/Wnt-k_100-paths.txt output/Wnt-k_100-ranked-edges.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Post results to GraphSpace\n",
    "This step is not too difficult if we just want to view the nodes, edges and paths on GraphSpace. However, there is much more information we have that we would like to be able to quickly access and see on GraphSpace. For example:\n",
    "  - What sources of evidence do we have for each interaction in the top 100 paths?\n",
    "  - Which proteins and PPIs in the top 100 paths are known to be part of the NetPath or KEGG WNT pathway?\n",
    "  \n",
    "If you want to see the main code where we build the GraphSpace graph and post the graph, skip to the bottom."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# now import the packages we're going to use below\n",
    "from graphspace_python.api.client import GraphSpace\n",
    "from graphspace_python.graphs.classes.gsgraph import GSGraph\n",
    "import networkx as nx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read in the PathLinker results and keep track of the k for each of the edges\n",
    "ranked_edges = {}\n",
    "with open('output/Wnt-k_100-ranked-edges.txt', 'r') as file_handle:\n",
    "    for line in file_handle:\n",
    "        if line[0] == '#':\n",
    "            continue\n",
    "        line = line.rstrip().split('\\t')\n",
    "        edge = (line[0], line[1])\n",
    "        k = line[2]\n",
    "        ranked_edges[edge] = k\n",
    "        \n",
    "# Read in the sources and targets file so we can change the shape and color of the sources and targets\n",
    "sources = set()\n",
    "targets = set()\n",
    "with open('data/Wnt-sources-targets.tsv', 'r') as file_handle:\n",
    "    for line in file_handle:\n",
    "        if line[0] == '#':\n",
    "            continue\n",
    "        prot, prot_type = line.rstrip().split('\\t')[0:2]\n",
    "        if prot_type.lower() in [\"receptor\", \"source\"]:\n",
    "            sources.add(prot)\n",
    "        elif prot_type.lower() in [\"tf\", \"target\"]:\n",
    "            targets.add(prot)\n",
    "\n",
    "print(\"%d edges in PathLinker output, %d sources, %d targets\" % (len(ranked_edges), len(sources), len(targets)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# before adding these edges to a GraphSpace object, \n",
    "# we need to get the node, edge, and pathway information we want to post\n",
    "# we also need to be able to map the uniprot IDs to their more friendly gene names\n",
    "uniprot_to_gene = {}\n",
    "with open('data/2017_06-human-interactome-mapping.tsv', 'r') as file_handle:\n",
    "    for line in file_handle:\n",
    "        if line[0] == '#':\n",
    "            continue\n",
    "        uniprotID, gene_name = line.rstrip().split('\\t')\n",
    "        uniprot_to_gene[uniprotID] = gene_name\n",
    "        \n",
    "# get the edge weights from the interactome. We will add these to the popups later\n",
    "edge_weights = {}\n",
    "with open('data/2017-06-16-human-ppi-weighted-cap0_75.tsv', 'r') as file_handle:\n",
    "    for line in file_handle:\n",
    "        if line[0] == '#':\n",
    "            continue\n",
    "        u, v, w, evidence = line.rstrip().split('\\t')\n",
    "        edge_weights[(u,v)] = float(w)\n",
    "        \n",
    "print(\"%s interactome edges\" % (len(edge_weights)))\n",
    "        \n",
    "# get the netpath and kegg edges and nodes from the evidence file\n",
    "# KEGG WNT pathway: http://www.genome.jp/kegg-bin/show_pathway?hsa04310\n",
    "# NetPath WNT pathway: http://www.netpath.org/pathways?path_id=NetPath_8\n",
    "kegg_wnt_edges = set()\n",
    "netpath_wnt_edges = set()\n",
    "evidence_file = 'data/2017_06-human-interactome-evidence.tsv'\n",
    "with open(evidence_file, 'r') as file_handle:\n",
    "    for line in file_handle:\n",
    "        if line[0] == \"#\":\n",
    "            continue\n",
    "        u,v, directed, interactiontype, detectionmethod, pubid, source = line.rstrip().split('\\t')\n",
    "        if source.lower() == 'kegg' and pubid.lower() == 'kegg:hsa04310':\n",
    "            kegg_wnt_edges.add((u,v))\n",
    "            if directed == \"False\":\n",
    "                kegg_wnt_edges.add((v,u))\n",
    "        elif source.lower() == 'netpath' and pubid.lower() == \"netpath:netpath_8\":\n",
    "            netpath_wnt_edges.add((u,v))\n",
    "            if directed == \"False\":\n",
    "                netpath_wnt_edges.add((v,u))\n",
    "kegg_wnt_nodes = set([n for u,v in kegg_wnt_edges for n in (u,v)])\n",
    "netpath_wnt_nodes = set([n for u,v in netpath_wnt_edges for n in (u,v)])\n",
    "\n",
    "print(\"%s kegg wnt nodes, %s kegg wnt edges\" % (len(kegg_wnt_nodes), len(kegg_wnt_edges)))\n",
    "print(\"%s netpath wnt nodes, %s netpath wnt edges\" % (len(netpath_wnt_nodes), len(netpath_wnt_edges)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# this next section is to get the sources of evidence, interaction type, and pubmed IDs for each interaction\n",
    "# I chose to load it into a multi-level dictionary organized by source database\n",
    "def getEvidence(edges, evidence_file):\n",
    "    \"\"\"\n",
    "    *edges*: a set of edges for which to get the evidence for\n",
    "    *split_family_nodes*: add evidence of family edges to pairwise interactions \n",
    "    *add_ev_to_family_edges*: add evidence of PPI edge covered by a family edge to the family edge. \n",
    "                              If this option is specified, then a dictionary of direct evidence along with\n",
    "                              a dictionary of the ppi evidence applied to the family edge is returned\n",
    "    returns a multi-level dictionary with the following structure\n",
    "    edge: \n",
    "      db/source: \n",
    "        interaction_type: \n",
    "          detection_method: \n",
    "            publication / database ids\n",
    "    for NetPath, KEGG and SPIKE, the detection method is the pathway name \n",
    "    NetPath, KEGG, and Phosphosite also have a pathway ID in the database/id set\n",
    "    which follows convention of id_type:id (for example: kegg:hsa04611)\n",
    "    \"\"\"\n",
    "    # initialize the evidence, edge_types, and edge dir/undir dictionaries\n",
    "    evidence = {}\n",
    "    edge_types = {}\n",
    "    edge_dir = {}\n",
    "\n",
    "    print(\"Reading evidence file %s\" % (evidence_file))\n",
    "\n",
    "    # create a graph of the passed in edges \n",
    "    G = nx.Graph()\n",
    "    G.add_edges_from(edges)\n",
    "\n",
    "    # initialize the dictionaries\n",
    "    for t,h in G.edges():\n",
    "        evidence[(t,h)] = {}\n",
    "        evidence[(h,t)] = {}\n",
    "        edge_types[(t,h)] = set()\n",
    "        edge_types[(h,t)] = set()\n",
    "        edge_dir[(t,h)] = False\n",
    "        edge_dir[(h,t)] = False\n",
    "\n",
    "    file_handle = open(evidence_file, 'r')\n",
    "    for line in file_handle:\n",
    "        if line[0] == \"#\":\n",
    "            continue\n",
    "        u,v, directed, interactiontype, detectionmethod, pubid, source = line.rstrip().split('\\t')\n",
    "        # header line of file\n",
    "        #uniprot_a  uniprot_b   directed    interaction_type    detection_method    publication_id  source\n",
    "        directed = True if directed == \"True\" else False\n",
    "\n",
    "        # We only need to get the evidence for the edges passed in, so if this edge is not in the list of edges, skip it\n",
    "        # G is a Graph so it handles undirected edges correctly\n",
    "        if not G.has_edge(u,v):\n",
    "            continue\n",
    "\n",
    "        evidence = addToEvidenceDict(evidence, (u,v), directed, source, interactiontype, detectionmethod, pubid)\n",
    "        edge_types = addEdgeType(edge_types, (u,v), directed, source, interactiontype)\n",
    "\n",
    "        if directed is True:\n",
    "            edge_dir[(u,v)] = True\n",
    "        else:\n",
    "            # if they are not already directed, then set them as undirected\n",
    "            if (u,v) not in edge_dir:\n",
    "                edge_dir[(u,v)] = False\n",
    "                edge_dir[(v,u)] = False\n",
    "\n",
    "    return evidence, edge_types, edge_dir\n",
    "\n",
    "\n",
    "    \n",
    "def addToEvidenceDict(evidence, e, directed, source, interactiontype, detectionmethod, pubid):\n",
    "    \"\"\" add the evidence of the edge to the evidence dictionary\n",
    "    *pubids*: publication id to add to this edge. \n",
    "    \"\"\"\n",
    "    if source not in evidence[e]:\n",
    "        evidence[e][source] = {}\n",
    "    if interactiontype not in evidence[e][source]:\n",
    "        evidence[e][source][interactiontype] = {}\n",
    "    if detectionmethod not in evidence[e][source][interactiontype]:\n",
    "        evidence[e][source][interactiontype][detectionmethod] = set()\n",
    "    evidence[e][source][interactiontype][detectionmethod].add(pubid)\n",
    "\n",
    "    if not directed:\n",
    "        # add the evidence for both directions\n",
    "        evidence = addToEvidenceDict(evidence, (e[1],e[0]), True, source, interactiontype, detectionmethod, pubid)\n",
    "\n",
    "    return evidence\n",
    "\n",
    "\n",
    "def addEdgeType(edge_types, e, directed, source, interactiontype):\n",
    "    if e not in edge_types:\n",
    "        edge_types[e] = set()\n",
    "    # add the edge type as well\n",
    "    # direction was determined using the csbdb_interface.psimi_interaction_direction dictionary in CSBDB. \n",
    "    if not directed:\n",
    "        # it would be awesome if we knew which edges are part of complex formation vs other physical interactions\n",
    "        # is there an mi term for that?\n",
    "        t,h = e\n",
    "        edge_types[(t,h)].add('physical')\n",
    "        edge_types[(h,t)].add('physical')\n",
    "\n",
    "    elif source == \"SPIKE\":\n",
    "        edge_types[e].add('spike_regulation')\n",
    "\n",
    "    elif source == \"KEGG\":\n",
    "        if \"phosphorylation\" in interactiontype and \"dephosphorylation\" not in interactiontype:\n",
    "            # Most of them are phosphorylation\n",
    "            edge_types[e].add('phosphorylation')\n",
    "        if \"activation\" in interactiontype:\n",
    "            edge_types[e].add('activation')\n",
    "        if \"inhibition\" in interactiontype:\n",
    "            edge_types[e].add('inhibition')\n",
    "        else:\n",
    "            edge_types[e].add('enzymatic')\n",
    "    else:\n",
    "        # the rest is psi mi tags\n",
    "        # MI:0217 is for phosphorylation\n",
    "        if \"MI:0217\" in interactiontype:\n",
    "            # Most of the directed edges are phosphorylation\n",
    "            edge_types[e].add('phosphorylation')\n",
    "        else:\n",
    "            # TODO for now, just call the rest of the directed edges enzymatic. \n",
    "            edge_types[e].add('enzymatic')\n",
    "\n",
    "    return edge_types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a small example of what the evidence dictionary looks like. I chose this edge because it is part of the NetPath WNT pathway and has a lot of support from multiple databases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# example edge: GSK3B-APC\n",
    "edges = [(\"P49841\", \"P25054\")]\n",
    "evidence_file = 'data/2017_06-human-interactome-evidence.tsv'\n",
    "evidence, edge_types, edge_dir = getEvidence(edges, evidence_file)\n",
    "e = edges[0]\n",
    "print(\"Evidence for edge %s-%s:\" % (e[0], e[1]))\n",
    "for source in evidence[e]:\n",
    "    print(source)\n",
    "    # for databases that don't provide an interaction type, I left it empty\n",
    "    for interactiontype in evidence[e][source]:\n",
    "        print(\"\\t%s\" % interactiontype)\n",
    "        # for curated databases or databases that don't provide a detection method, I put the name of the pathway or DB for the detection method\n",
    "        for detectionmethod in evidence[e][source][interactiontype]:\n",
    "            print(\"\\t\\t%s\" % detectionmethod)\n",
    "            for pubid in evidence[e][source][interactiontype][detectionmethod]:\n",
    "                print(\"\\t\\t\\t%s\" % pubid)\n",
    "print(\"Edge types:\", edge_types[e])\n",
    "print(\"Edge is directed:\", edge_dir[e])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# converts the evidence dictionary to html for the popup\n",
    "def evidenceToHTML(u,v,evidence):\n",
    "    annotation = '<dl>'\n",
    "    sources = sorted(evidence.keys())\n",
    "    for source in sources:\n",
    "        annotation += '<dt>%s</dt>' % (source)\n",
    "        # TODO add interaction type color\n",
    "        for interactiontype in evidence[source]:\n",
    "            if interactiontype != '' and interactiontype != \"None\":\n",
    "                # use bull instead of li to save on white space\n",
    "                # nbsp stands for non-breaking space\n",
    "                annotation += '&bull;&nbsp&nbsp%s <br>' % interactiontype\n",
    "            for detectionmethod in evidence[source][interactiontype]:\n",
    "                # add a bullet point here after 4 spaces\n",
    "                annotation += '&nbsp&nbsp&nbsp&nbsp&bull;&nbsp&nbsp'\n",
    "                annotation += '%s  ' % detectionmethod\n",
    "\n",
    "                # now add the pubmed IDs. &nbsp is the html for a non-breaking space\n",
    "                pub_ids = evidence[source][interactiontype][detectionmethod]\n",
    "                #KEGG doesn't have pub ids. It has a pathway map and entry (evidence)\n",
    "                # now get the html for each of the links\n",
    "                pub_ids = [parsePubID(pub_id) for pub_id in pub_ids if parsePubID(pub_id) != '']\n",
    "\n",
    "                # use a non-breaking space with a comma so they all stay on the same line\n",
    "                annotation += ',&nbsp'.join(pub_ids)\n",
    "                annotation += \"<br>\"\n",
    "        annotation += '<br>'\n",
    "    return annotation\n",
    "\n",
    "# adds the links to the pubmed and other IDs\n",
    "def parsePubID(publication_id):\n",
    "    id_type, pubid = publication_id.split(':')\n",
    "    if id_type == 'pubmed':\n",
    "        pubmedurl = 'http://www.ncbi.nlm.nih.gov/pubmed/%s' % (pubid)\n",
    "        desc = '<a style=\"color:blue\" href=\"%s\" target=\"PubMed\">pmid:%s</a>' % (pubmedurl,pubid)\n",
    "    elif id_type == 'phosphosite':\n",
    "        phosphourl = 'http://www.phosphosite.org/siteAction.action?id=%s' % (pubid)\n",
    "        desc = '<a style=\"color:blue\" href=\"%s\" target=\"PSP\">phosphosite:%s</a>' % (phosphourl,pubid)\n",
    "    elif id_type == 'kegg':\n",
    "        # links to KEGG pathway map\n",
    "        kegg_map_link = 'http://www.kegg.jp/kegg-bin/show_pathway?'\n",
    "        # links to KEGG pathway entry (evidence)\n",
    "        kegg_entry_link = 'http://www.kegg.jp/dbget-bin/www_bget?pathway+'\n",
    "        pathway_map_link = '<a style=\"color:blue\" href=\"%s%s\" target=\"KEGG\">map</a>' % (kegg_map_link,pubid)\n",
    "        pathway_entry_link = '<a style=\"color:blue\" href=\"%s%s\" target=\"KEGG\">evidence</a>' % (kegg_entry_link,pubid)\n",
    "        desc = \"%s,&nbsp%s\"%(pathway_map_link,pathway_entry_link)\n",
    "    elif id_type == 'netpath':\n",
    "        netpath_url = \"http://www.netpath.org/reactions?path_id=%s\" % (pubid)\n",
    "        # links to KEGG pathway entry (evidence)\n",
    "        desc = '<a style=\"color:blue\" href=\"%s\" target=\"NetPath\">netpath:%s</a>' % (netpath_url, pubid)\n",
    "    else:\n",
    "        # skip the rest for now\n",
    "        desc = ''\n",
    "\n",
    "    return desc\n",
    "\n",
    "\n",
    "def getMainEdgeType(u,v, edge_types):\n",
    "    \"\"\" a single edge can have multiple edge types according to the different sources or databases\n",
    "    Choose a main edge type here\n",
    "    *edge_types* the set of edge types for a given edge \n",
    "    \"\"\"\n",
    "    main_edge_type = None\n",
    "    edge_type_order = ['phosphorylation', 'enzymatic', 'spike_regulation', 'activation', 'inhibition', 'physical']\n",
    "    for edge_type in edge_type_order:\n",
    "        if edge_type in edge_types:\n",
    "            main_edge_type = edge_type \n",
    "            break\n",
    "            \n",
    "    if main_edge_type is None:\n",
    "        print(\"Warning: edge type of %s->%s not found. Setting to activation\" % (u,v))\n",
    "        main_edge_type = 'activation'\n",
    "\n",
    "    return main_edge_type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# takes in a node ID and the list of paths its in, and returns the html popup\n",
    "def makeNodePopup(n, pathswithnode):\n",
    "    htmlstring = ''\n",
    "    uniproturl = 'http://www.uniprot.org/uniprot/%s' \n",
    "    htmlstring += '<b>Uniprot ID</b>: <a style=\"color:blue\" href=\"%s\" target=\"UniProtKB\">%s</a><br>' % (uniproturl%n, n)\n",
    "    htmlstring += '<hr />'\n",
    "\n",
    "    htmlstring += '<b>Paths</b>: %s<br>' %(','.join(str(k) for k in sorted(pathswithnode)))\n",
    "    \n",
    "    return htmlstring\n",
    "\n",
    "# takes in the u->v edge, first path index k its in, and sources of evidence \n",
    "# returns the html edge popup\n",
    "def makeEdgePopup(u, v, k, evidence):\n",
    "    annotation = '' \n",
    "    annotation += '<b>%s - %s</b></br>'%(uniprot_to_gene[u], uniprot_to_gene[v])\n",
    "    annotation += '<b>%s - %s</b></br>'%(u,v)\n",
    "    annotation += '<b>Weight</b>: %.3f</br>' % (edge_weights[(u,v)])\n",
    "    annotation += '<b>Edge Ranking</b>: %s' % (k)\n",
    "\n",
    "    annotation += '<hr /><h><b>Sources of Evidence</b></h>'\n",
    "    annotation += evidenceToHTML(u,v,evidence[(u,v)])\n",
    "\n",
    "    return annotation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is an example of what the node and edge popups will look like for the node APC and the edge GSK3B-APC:\n",
    "\n",
    "APC popup | GSK3B-APC popup\n",
    "--------- | ---------------\n",
    "![popup](images/popup-apc.png) | ![popup](images/popup-gsk3b-apc.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Dictionaries of node and edge properties\n",
    "NODE_COLORS = {\n",
    "        #'target'  : '#FFFF60',  # yellow\n",
    "        #'source'  : '#8CE1DC',  # light blue\n",
    "        'target'  : '#4286f4',  # blue\n",
    "        'source'  : '#4286f4',  # blue\n",
    "        'default' : '#D8D8D8',  # gray\n",
    "        'kegg'    : '#ad6cfc',  # purple\n",
    "        'netpath' : '#4286f4',  # blue\n",
    "}\n",
    "NODE_SHAPES = {\n",
    "        'source'  : 'triangle',\n",
    "        'target'  : 'rectangle',\n",
    "        'default' : 'ellipse',\n",
    "}\n",
    "EDGE_COLORS = {\n",
    "        'physical'        : '#27AF47',  # green\n",
    "        'phosphorylation' : '#F07406',  # orange\n",
    "        #'enzymatic'       : '#2A69DC',  # blue\n",
    "        'enzymatic'       : '#DD4B4B',  # red\n",
    "        'activation'      : 'grey',\n",
    "        'inhibition'      : 'grey',\n",
    "        'spike_regulation': 'brown',\n",
    "        'kegg'            : '#ad6cfc', # purple\n",
    "        'netpath'         : '#4286f4',  # blue\n",
    "}\n",
    "\n",
    "# Adds all of the nodes and edges to the GraphSpace object with the specified shapes and colors\n",
    "def constructGraph(ranked_edges, sources, targets):\n",
    "    '''\n",
    "    Posts the pathlinker result to graphspace\n",
    "\n",
    "    :param sources: list of source nodes\n",
    "    :param targets: list of target nodes\n",
    "    '''\n",
    "    # NetworkX object\n",
    "    #G = nx.DiGraph()\n",
    "    G = GSGraph()\n",
    "    \n",
    "    # first get the evidence, edge types, and edge directionality for all of the edges we will be posting\n",
    "    evidence, edge_types, edge_dir = getEvidence(ranked_edges.keys(), evidence_file)\n",
    "\n",
    "    # get the nodes from the set of edges. we'll add those first\n",
    "    nodes = set([n for u,v in ranked_edges for n in (u,v)])\n",
    "\n",
    "    # add GraphSpace/Cytoscape.js attributes to all nodes.\n",
    "    for n in nodes:\n",
    "        #default is gray circle\n",
    "        node_type = 'default'\n",
    "        if n in sources:\n",
    "            # if n is the source, make it a triangle\n",
    "            node_type = 'source'\n",
    "        elif n in targets:\n",
    "            # if n is a taret, make it a square\n",
    "            node_type = 'target'\n",
    "\n",
    "        # find the path ranks of all of the paths the node is in so we can set the k value of this node\n",
    "        pathswithnode = set([int(ranked_edges[(t,h)]) for t,h in ranked_edges if t==n or h==n])\n",
    "        # The k value is used by the rank filter on GraphSpace\n",
    "        # All nodes and edges need a value for the filter to work\n",
    "        # Here we are filtering by the first path the node appears in which will allow us to step through the paths on GraphSpace\n",
    "        k_value = min(pathswithnode)\n",
    "        # set the name of the node to be the gene name and add the k to the label\n",
    "        gene_name = uniprot_to_gene[n]\n",
    "        node_popup = makeNodePopup(n, pathswithnode)\n",
    "        label = \"%s\\n%d\"%(gene_name,k_value)\n",
    "\n",
    "        G.add_node(gene_name, popup=node_popup, label=label, k=k_value)\n",
    "\n",
    "        # now add the style for the node\n",
    "        shape = NODE_SHAPES[node_type]\n",
    "        color = NODE_COLORS[node_type]\n",
    "        if n in netpath_wnt_nodes:\n",
    "            color = NODE_COLORS['netpath']\n",
    "        elif n in kegg_wnt_nodes:\n",
    "            color = NODE_COLORS['kegg']\n",
    "        # some attributes (such as opacity) are not implemented in graphspace-python. \n",
    "        # For those, we need to make our own dictionary of the attribute and value\n",
    "        # for more style settings, see http://js.cytoscape.org/#style\n",
    "        attr_dict = {}\n",
    "        #attr_dict['background-opacity'] = 0.8\n",
    "\n",
    "        G.add_node_style(gene_name, attr_dict=attr_dict, shape=shape, color=color, width=45, height=45,\n",
    "                         style='solid', border_color=color, border_width=2, bubble=color)\n",
    "\n",
    "    # Add all of the edges and their Graphspace/Cytoscape.js attributes\n",
    "    for (u,v) in ranked_edges:\n",
    "        # get the main edge type so we can style and color edges accordingly\n",
    "        main_edge_type = getMainEdgeType(u,v,edge_types[(u,v)])\n",
    "\n",
    "        gene_name_u = uniprot_to_gene[u]\n",
    "        gene_name_v = uniprot_to_gene[v]\n",
    "        k_value = ranked_edges[(u,v)]\n",
    "        edge_popup = makeEdgePopup(u,v,k_value, evidence)\n",
    "\n",
    "        G.add_edge(gene_name_u, gene_name_v, directed=edge_dir[(u,v)], popup=edge_popup, k=k_value)\n",
    "\n",
    "        color = EDGE_COLORS[main_edge_type]\n",
    "        if (u,v) in netpath_wnt_edges:\n",
    "            color = EDGE_COLORS['netpath']\n",
    "        elif (u,v) in kegg_wnt_edges:\n",
    "            color = EDGE_COLORS['kegg']\n",
    "        # TODO use the edge weight to set the width of the edge\n",
    "        width = 2\n",
    "        arrow_shape = None\n",
    "        if edge_dir[(u,v)] is True:\n",
    "            arrow_shape = \"triangle\"\n",
    "        # if this is an inhibition edge, make the arrowhead be a T shape\n",
    "        if 'activation' not in edge_types[(u,v)] and 'inhibition' in edge_types[(u,v)]:\n",
    "            arrow_shape = 'tee'       \n",
    "        # some attributes (such as opacity) are not implemented in graphspace-python. \n",
    "        # For those, we need to make our own dictionary of the attribute and value\n",
    "        # for more style settings, see http://js.cytoscape.org/#style\n",
    "        attr_dict = {}\n",
    "        #attr_dict['opacity'] = 0.8\n",
    "\n",
    "        G.add_edge_style(gene_name_u, gene_name_v, attr_dict=attr_dict,\n",
    "                         directed=edge_dir[(u,v)], color=color, width=width, \n",
    "                         arrow_shape=arrow_shape, edge_style='solid')\n",
    "    return G"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have all of the functions for getting the evidence and popups setup, we're ready to build our GraphSpace graph and then post it to GraphSpace.\n",
    "\n",
    "For more info about using the Graphspace Python library, see the [documentation](http://manual.graphspace.org/projects/graphspace-python/en/latest/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# now finally construct the GraphSpace graph and post it to GraphSpace!\n",
    "G = constructGraph(ranked_edges, sources, targets)\n",
    "\n",
    "# TODO add a description of the graph with a legend describing shapes and colors\n",
    "desc = ''\n",
    "title = 'NetPath WNT Pathway PathLinker Reconstruction'\n",
    "metadata = {'description':desc, 'title':title}\n",
    "G.set_data(metadata)\n",
    "tags = ['pathlinker', 'netpath', '2017icsb', 'tutorial']\n",
    "G.set_tags(tags)\n",
    "\n",
    "graph_name = \"netpath-wnt-pathlinker-k100\"\n",
    "G.set_name(graph_name)\n",
    "    \n",
    "# post to graphspace\n",
    "# set your username and password here\n",
    "username = 'user6@example.com'\n",
    "password = 'user6'\n",
    "gs = GraphSpace(username, password)\n",
    "gs_graph = gs.get_graph(graph_name, owner_email=username)\n",
    "if gs_graph is None:\n",
    "    print(\"\\nPosting graph '%s' to graphspace\\n\" % (graph_name))\n",
    "    gs_graph = gs.post_graph(G)\n",
    "else:\n",
    "    print(\"\\nGraph '%s' already exists. Updating it\\n\" % (graph_name))\n",
    "    # this can take a while if your graph has a lot of nodes and edges \n",
    "    gs_graph = gs.update_graph(G, graph_name=graph_name, owner_email=username)\n",
    "    print(\"Done\")\n",
    "    print(gs_graph.url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from graphspace_python.graphs.classes.gsgroup import GSGroup\n",
    "# you can also share your graph with a group\n",
    "# create the group if it doesn't exist\n",
    "#group = gs.post_group(GSGroup(name='icsb2017', description='sample group'))\n",
    "# or get the group you already created it\n",
    "#group = gs.get_group(name='icsb2017')\n",
    "#print(group.url)\n",
    "#gs.share_graph(graph=G, group=group)\n",
    "gs_graph = gs.publish_graph(graph=G)\n",
    "print(gs_graph.url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reconstruction of the WNT Signaling Pathway\n",
    "Here is the layout I made for our graph:\n",
    "![NetPath-WNT-PathLinker-k100](images/NetPath-WNT-PathLinker-k100.png)\n",
    "\n",
    "- First non-WNT pathay node is CFTR\n",
    "  - Experimentally validated RYK-CFTR-DAB2 WNT signal\n",
    "  - Compare to figure 3b in their paper\n",
    "  - Ritz et al., [Pathways on demand: automated reconstruction of human signaling networks](https://www.nature.com/articles/npjsba20162). npj: Systems Biology and Applications, 2016.\n",
    "- We could probably use different edge colors so that the NetPath and KEGG edges stand out more. Maybe blue for NetPath, Orange for KEGG and gray for everything else.\n",
    "- Below I have the code for updating a graph (if you make changes above) while keeping the node positions of a layout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Currently layouts store style attributes as well as x and y positions\n",
    "# so if you layed out the graph how you wanted it but now want to update the style (such as edge width), \n",
    "# you will need to copy the x and y positions of the layout you made to the updated graph\n",
    "# I created the layout 'layout1', so I'll use that\n",
    "layout_name = 'layout1'\n",
    "layout = gs.get_graph_layout(graph=gs_graph,layout_name=layout_name)\n",
    "# set the x and y position of each node in the updated graph to the x and y positions of the layout you created\n",
    "print(\"Setting the x and y coordinates of each node to the positions in %s\" % layout_name)\n",
    "for node, positions in layout.positions_json.items():\n",
    "    G.set_node_position(node_name=node, x=positions['x'], y=positions['y'])\n",
    "    \n",
    "# now re-post the graph and the positions should be set\n",
    "print(\"Updating graph\", graph_name)\n",
    "gs_graph = gs.update_graph(G, graph_name=graph_name, owner_email=username)\n",
    "# have to publish/share again after updating\n",
    "gs_graph = gs.publish_graph(graph=G)\n",
    "print(\"Done\")\n",
    "print(gs_graph.url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Post your own networks!\n",
    "\n",
    "You should have all the tools you need to post your own networks to GraphSpace.\n",
    "\n",
    "If you run into any problems, we'd be happy to help! Send us an email or if there's a bug in the code, please open an issue in the appropriate GitHub repository describing the error and the steps to reproduce it.\n",
    "- https://github.com/adbharadwaj/graphspace-python\n",
    "- https://github.com/Murali-group/GraphSpace\n",
    "\n",
    "Happy posting!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}