Por chispa pequeña nota al azar

1. Obtener datos de un archivo hdfs

    val hdfs=org.apache.hadoop.fs.FileSystem.get(new java.net.URI("hdfs://hacluster"),
        new org.apache.hadoop.conf.Configuration())
    val fSDataInputStream1=hdfs.open(new Path(hdfs://hacluster/A/B/test.txt))
    val bufferedReader1=new BufferedReader(new InputStream(fSDataInputStream1))
    val line=bufferedReader1.readLine()

2. Definir una función para crear SSC

     val sc = SparkContext.getOrCreate()
     def funCreateStreamingContext():StreamingContext={
          val newSsc= new StreamingContext(sc,Seconds(60))
          println("Creating new StreamingContext")
          newSsc.chekpoint(vCheckPoint)
          newSsc
     }

3. Cree SSC

    val checkPointPath ="hdfs://hacluster/A/B/checkPointPath"
    val ssc=StreamingContext.getActiveOrCreate(checkPointPath ,funCreateStreamingContext)