spark command line parsing with class scopt.OptionParser not found

Question 1: The third-party package scopt is used when parsing the spark-submit command line parameters, which needs to be imported

<dependency>
  <groupId>com.github.scopt</groupId>
  <artifactId>scopt_2.10</artifactId>
  <version>3.2.0</version>
</dependency>

, after the introduction, use the idea package (build-build artifacts-rebuild) directly to report an error: the class scopt.OptionParser cannot be found

Then realized that the dependencies were not packaged

Reconfigure the file-project structure to add all the packages, repackage them, and still report the same error

Then tried to package using maven

configure first

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>

  <configuration>
      <archive>
        <manifest>
          <mainClass>com.daojia.xing.SuYunSDK</mainClass>
        </manifest>
      </archive>
      <descriptorRefs>
        <descriptorRef>jar-with-dependencies</descriptorRef>
      </descriptorRefs>
    <descriptorRefs>
      <descriptorRef>jar-with-dependencies</descriptorRef>
    </descriptorRefs>
  </configuration>
</plugin>

Click the bottom left icon to pop up the pull-up box, a single-machine maven project dialog box appears on the left, and then plugins select assembly: assembly

problem solved.

ps：

Question 2: The hive.hiveContext class is used in the program and needs to be introduced. The wrong package was introduced before, causing the program to prompt an error

The correct import is as follows:

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-hive_2.10</artifactId>
  <version>1.6.1</version>
</dependency>

Question 3: It's another weird question. My knowledge is shallow. (String, String) I don't know what the hell this is in scala. It's not an array, it's not a list, it's not a map. Finally, I found that scala has a type called Yuanzu. Here Tangled for a long time, to strengthen learning

 

scala> val types=hiveContext.sql("select * from a_fin_account_day limit 10").dtypes
types: Array[(String, String)] = Array((work_date,StringType), (user_id,LongType), (account_id,LongType), (source,StringType), (sy_vip_flag,StringType), (sy_vip_account_type,StringType), (dk_bussiness,DoubleType), (balance,DoubleType), (recharge,DoubleType), (recharge_num,DoubleType), (consume,DoubleType), (consume_num,DoubleType), (order_balance_pay,DoubleType), (order_refund,DoubleType), (account_refund,DoubleType), (adjust,DoubleType), (is_new_charge,DoubleType), (dk_jz_consume,DoubleType), (dk_lr_consume,DoubleType), (bi_update_time,StringType))
scala> val line=types(0)
line: (String, String) = (work_date,StringType)
scala> line(0)
<console>:41: error: (String, String) does not take parameters
              line(0)
                ^
scala> line._1
res28: String = work_date