题目：写一个java程序，统计一下news.txt里面的词频。要求程序输出出现次数排名前10位的单词。请用N（N作为参数传入）个线程统计，每个线程统计文章的1/N的内容。
public class WordFrequencyCount {
public static void wordFrequencyCount(int n) throws ExecutionException, InterruptedException {
File wf = new File("news.txt");
int taskNum = n;
// 创建一个线程池
ExecutorService pool = Executors.newFixedThreadPool(taskNum);
// 创建多个有返回值的任务
List<Future> list = new ArrayList<Future>();
for (int i = 0; i < taskNum; i++) {
Callable c = new MyCallable(wf, i/taskNum, (i+1)/taskNum*wf.length());
// 执行任务并获取Future对象
Future f = pool.submit(c);
list.add(f);
}
// 关闭线程池
pool.shutdown();
// 获取所有并发任务的运行结果
TreeMap tMap = new TreeMap();
for (Future f : list) {
HashMap hMap = (HashMap) f.get();
//对不同线程处理的结果进行整合
tMap.putAll(hMap);
}
//打印输出，查看结果
//Iterator iterator = tMap.entrySet().iterator();
List<Map.Entry<String, Integer>> entrieList = new ArrayList<Map.Entry<String, Integer>>(tMap.entrySet());
//通过比较器实现排序
Collections.sort(entrieList, new Comparator<Map.Entry<String, Integer>>() {
//降序排序
public int compare(Map.Entry<String, Integer> o1, Map.Entry<String, Integer> o2) {
return o2.getValue().compareTo(o1.getValue());
}
});
for (int i = 0; i < 10; i++){
Map.Entry<String, Integer> map = entrieList.remove(i);
System.out.println(map.getKey() + ":" + map.getValue());
}
}
public static void main(String[] args) throws ExecutionException, InterruptedException {
wordFrequencyCount(10);
}
}
class MyCallable implements Callable<Object> {
private FileChannel fc;
private FileLock fl;
private MappedByteBuffer mbBuf;
private HashMap hm;
public MyCallable(File src, long start, long end) {
try {
//得到当前文件的通道
fc = new RandomAccessFile(src, "rw").getChannel();
//锁定当前文件的部分
fl = fc.lock(start, end, false);
//对当前文件片段建立内存映射，如果文件过大需要切割成多个片段
mbBuf = fc.map(FileChannel.MapMode.READ_ONLY, start, end);
//创建HashMap实例存放处理结果
hm = new HashMap<String, Integer>();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public Object call() throws Exception {
String str = Charset.forName("UTF-8").decode(mbBuf).toString();
//使用StringTokenizer分析单词
StringTokenizer token = new StringTokenizer(str);
String word;
while(token.hasMoreTokens()) {
//将处理结果放到一个HashMap中
word = token.nextToken();
if(null != hm.get(word)) {
hm.put(word, (Integer)hm.get(word)+1);
} else {
hm.put(word, 1);
}
}
try {
//释放文件锁
fl.release();
} catch (IOException e) {
e.printStackTrace();
}
return hm;
}
}
输出结果：
the:51
of:25
and:20
from:13
said:13
on:11
are:9
their:9
Nielsen:8
at:8
附 news.txt内容如下：
Facing a rising tide of outrage from Democrats and some Republicans over the forced separation of migrant children and parents at the U.S.-Mexico border, U.S. President Donald Trump dug in Monday, while his top law enforcement officials also forcefully defended the practice. "The United States will not be a migrant camp and it will not be a refugee-holding facility — not on my watch," said Trump from the White House. "A country without borders is not a country at all," he said. Trump's defence followed Homeland Security Secretary Kirstjen Nielsen and Attorney General Jeff Sessions refusing to apologize for separating children from their parents at the National Sheriffs' Association on Monday in New Orleans. Speaking at a White House briefing Monday evening after her appearance in New Orleans, Nielsen told reporters the current immigration issue has been growing for years and is the product of loopholes that she said has created an open border. At the briefing, she again urged Congress to change laws, saying that her department was tasked with enforcing the laws. There is no law mandating the separation of children and parents at the border, but Nielsen said Monday that past administrations asked immigration agents to look the other way when families crossed the border illegally. Children can't go to jail with their parents, so they are separated. "The children are not being used as a pawn," Nielsen said in response to a question about accusations from Democrats. "We are trying to protect the children, which is why I'm asking Congress to act." Both the House and the Senate are controlled by Republicans. Nancy Pelosi, House minority leader, said on Twitter that what's happening now isn't an immigration issue but a humanitarian one, saying "we must do more to end the suffering of these young children who've been torn away from their parents." Sessions and other officials said the goal was to deter people from crossing illegally. The U.S. has been confronting an increase in illegal crossing attempts so far this year, but the levels are still far off those from the late 1980s until the early 2000s. Zero-tolerance announcement The zero-tolerance announcement, in fact, came just days after Trump himself touted statistics released by Customs and Border Enforcement showing that illegal crossings in 2017 had fallen to historic lows. Trump has continued to blame the Democrats for the separations in public comments and on social media. In a flurry of tweets on Monday morning before he spoke at the White House, he again cited concerns about gangs and crime rates in Central America, although those concerns have existed for several years, if not decades, and are what have prompted thousands to flee. In unrelated tweets about Europe's migrant crisis, Trump said he wouldn't allow the U.S. to be overwhelmed by the number of new arrivals that Germany has received. European nations, he said, had allowed "millions of people in who have so strongly and violently changed their culture." Germany's overall crime rate has fallen to record lows, according to a report in May, while its trend on violent crime is more unclear. The U.S. is significantly more violent than Germany with respect to homicide rates. Nielsen in her speech said that releasing parents with their children amounts to a "get out of jail free card" policy for illegal immigrants and accused some of posing as families to gain entry. Republican Sen. Susan Collins, of Maine, said she favours tighter border security, but expressed deep concerns about the child separation policy, as have some other Republicans, including House Speaker Paul Ryan of Wisconsin. "What the administration has decided to do is to separate children from their parents to try to send a message that if you cross the border with children, your children are going to be ripped away from you," said Collins. "That's traumatizing to the children who are innocent victims, and it is contrary to our values in this country." The president is also feeling the heat from numerous religious groups, including some conservative ones, a bloc that voted overwhelmingly for him in the 2016 presidential election. Influential Rev. Franklin Graham has called the policy "disgraceful," while a host of religious organizations including the Southern Baptist Convention, U.S. Conference of Catholic Bishops and the United Church of Christ have released statements in recent days condemning the separation of children from adult guardians. Despite Sessions's April announcement, and his frequent interviews and speeches since defending the approach — including an attempt to justify it by citing Bible passages — Nielsen a day earlier left many scratching their heads on Sunday evening by tweeting that the policy did not exist. Nielsen in subsequent tweets stressed that only in remarkable circumstances — endangerment of a child — are kids separated from their parents when people make an asylum claim at a legal port of entry. While Nielsen attempted to parse the issue, the administration is being challenged by its own policy, which entails additional facilities. According to one report last week, the government was even looking into the possibility of outdoor tent camps at military bases. Trump plans to meet with House Republicans on Tuesday to discuss pending immigration legislation amid an election-season debate over one of his favourite issues. The House is expected to vote this week on a bill pushed by conservatives that may not have enough support to pass, and a compromise measure with key proposals supported by the president. The White House has said Trump would sign either of those. Democratic congressman Adam Schiff of California said the administration is "using the grief, the tears, the pain of these kids as mortar to build our wall. And it's an effort to extort a bill to their liking in the Congress." Democratic Sen. Joe Donnelly of Indiana said on Sunday he is ready to co-sponsor legislation to end the practice. The House proposals face broad opposition from Democrats, and even if a bill does pass, the closely divided Senate seems unlikely to go along. The UN on Monday reiterated its condemnation of the U.S. separation policy, calling it