Java IO读写大文件的几种方式及测试

Java IO读写大文件的几种方式及测试

读取文件大小：1.45G
第一种，OldIO:

public static void oldIOReadFile() throws IOException{   
    BufferedReader br = new BufferedReader(new FileReader("G://lily_947.txt"));   
    PrintWriter pw = new PrintWriter("G://oldIO.tmp");   
    char[] c = new char[100*1024*1024];   
    for(;;){   
        if(br.read(c)!=-1){   
            pw.print(c);   
        }else{   
            break;   
        }   
    }   
    pw.close();   
    br.close();   
}  

耗时70.79s
第二种，newIO:

public static void newIOReadFile() throws IOException{   
        FileChannel read = new RandomAccessFile("G://lily_947.txt","r").getChannel();   
        FileChannel writer = new RandomAccessFile("G://newIO.tmp","rw").getChannel();   
        ByteBuffer bb = ByteBuffer.allocate(200*1024*1024);   
        while(read.read(bb)!=-1){   
            bb.flip();   
            writer.write(bb);   
            bb.clear();   
        }   
        read.close();   
        writer.close();   
           
    }  

耗时47.24s

第三种，RandomAccessFile:

public static void randomReadFile() throws IOException{   
        RandomAccessFile read = new RandomAccessFile("G://lily_947.txt","r");   
        RandomAccessFile writer = new RandomAccessFile("G://random.tmp","rw");   
        byte[] b = new byte[200*1024*1024];   
        while(read.read(b)!=-1){   
            writer.write(b);   
        }   
        writer.close();   
        read.close();   
    }  

耗时46.65

第四种，MappedByteBuffer：

public static void mappedBuffer() throws IOException{   
        FileChannel read = new FileInputStream("G://lily_947.txt").getChannel();   
        FileChannel writer = new RandomAccessFile("G://buffer.tmp","rw").getChannel();   
        long i = 0;   
        long size = read.size()/30;   
        ByteBuffer bb,cc = null;   
        while(i<read.size()&&(read.size()-i)>size){   
            bb = read.map(FileChannel.MapMode.READ_ONLY, i, size);   
            cc = writer.map(FileChannel.MapMode.READ_WRITE, i, size);   
            cc.put(bb);   
            i+=size;   
            bb.clear();   
            cc.clear();   
        }   
        bb = read.map(FileChannel.MapMode.READ_ONLY, i, read.size()-i);   
        cc.put(bb);   
        bb.clear();   
        cc.clear();   
        read.close();   
        writer.close();   
           
    }  

耗时：36

前三种读法对应的资源占用图如下：
相对于最后一种内存直接映射方式前面的测试其实无意义，基本秒杀。。。。。
对于很大的文件直接分块映射时内存会不够，这是因为MappedByteBuffer未被释放造成的，sun未提供直接回收MappedByteBuffer区域的方法，这个时候有两种方法解决，第一种比较愚笨的：

 System.gc();    
         System.runFinalization();    
         try {   
    Thread.sleep(3000);   
} catch (InterruptedException e) {   
       
    e.printStackTrace();   
}  

第二种网上找来的，利用反射调用clean方法：

public static void unmap(final MappedByteBuffer buffer) {   
        if (buffer == null) {   
            return;   
        }   
        AccessController.doPrivileged(new PrivilegedAction<Object>() {   
            public Object run() {   
                try {   
                    Method getCleanerMethod = buffer.getClass().getMethod("cleaner", new Class[0]);   
                    if (getCleanerMethod != null) {   
                        getCleanerMethod.setAccessible(true);   
                        Object cleaner = getCleanerMethod.invoke(buffer, new Object[0]);   
                        Method cleanMethod = cleaner.getClass().getMethod("clean", new Class[0]);   
                        if (cleanMethod != null) {   
                            cleanMethod.invoke(cleaner, new Object[0]);   
                        }   
                    }   
                } catch (Exception e) {   
                    e.printStackTrace();   
                }   
                return null;   
            }   
    
        });   
    }  

转载自 http://blog.sina.com.cn/s/blog_6699fade01012bvo.html