graphframes本地windows环境搭建

1.从github上下载graphframes包,或者从链接下载:https://download.csdn.net/download/weixin_42247685/10591772

2.解压后将graphframes-master\python下的graphframes文件夹拷贝到python/Lib/site-packages文件下。

3.在python中输入import  graphframes ,查看是否正常。

示例代码:

from graphframes import *
from pyspark.sql import SparkSession
import os

import graphframes

#下面这段必须有,否则报错,
os.environ["PYSPARK_SUBMIT_ARGS"] = (
    "--packages graphframes:graphframes:0.5.0-spark2.1-s_2.11 pyspark-shell"
)

sqlContext = SparkSession.builder.appName("Python Spark SQL basic example").config("spark.some.config.option", "some-value").getOrCreate()

v = sqlContext.createDataFrame([
  ("a", "Alice", 34),
  ("b", "Bob", 36),
  ("c", "Charlie", 30),
], ["id", "name", "age"])
# Create an Edge DataFrame with "src" and "dst" columns
e = sqlContext.createDataFrame([
  ("a", "b", "friend"),
  ("b", "c", "follow"),
  ("c", "b", "follow"),
], ["src", "dst", "relationship"])
# Create a GraphFrame

g = GraphFrame(v, e)

# Query: Get in-degree of each vertex.
g.inDegrees.show()

# Query: Count the number of "follow" connections in the graph.
g.edges.filter("relationship = 'follow'").count()

# Run PageRank algorithm, and show results.
results = g.pageRank(resetProbability=0.01, maxIter=20)
results.vertices.select("id", "pagerank").show()


猜你喜欢

转载自blog.csdn.net/weixin_42247685/article/details/81514169
今日推荐