I encountered a problem: the need for all requests for signature verification to prevent brush interfaces; incoming request url and body generates a text string as a header to the service end; already have an existing signature verification method String doSignature(String url, byte[] body);is currently based network library com.squareup.okhttp3:okhttp:3.14.2.
It's very simple, of course, is to write a interceptorthen url and body request object has passed like so:
public class SignInterceptor implements Interceptor {
@NonNull
@Override
public Response intercept(@NonNull Chain chain) throws IOException {
Request request = chain.request();
RequestBody body = request.body();
byte[] bodyBytes = null;
if (body != null) {
final Buffer buffer = new Buffer();
body.writeTo(buffer);
bodyBytes = buffer.readByteArray();
}
Request.Builder builder = request.newBuilder();
HttpUrl oldUrl = request.url();
final String url = oldUrl.toString();
final String signed = doSignature(url, bodyBytes));
if (!TextUtils.isEmpty(signed)) {
builder.addHeader(SIGN_KEY_NAME, signed);
}
return chain.proceed(builder.build());
}
}the okhttp ReqeustBodyis an abstract class, only the content output writeTomethod, to a content of the writing BufferedSinkinterface in the body, and then converted into the data byte[]that is a memory array. purpose can be achieved only class Bufferthat implements BufferedSinkan interface and converted to provide method memory array readByteArray. it looks nothing issues, can cause OOM?
Yes, depending on the type of request, if it is the interface to upload a file? If this file is large it? Upload interfaces may be used in public static RequestBody create(final @Nullable MediaType contentType, final File file)the method, if the file is for the body to achieve its writeToapproach is sink.writeAll(source);that we pass the method signature when used Buffer.readByteArrayare all the contents of the buffer memory turned into an array, which means that all the contents of the file to be turned into a memory array, that is likely to cause OOM at this moment! RequestBody.createsource code as follows:
public static RequestBody create(final @Nullable MediaType contentType, final File file) {
if (file == null) throw new NullPointerException("file == null");
return new RequestBody() {
@Override public @Nullable MediaType contentType() {
return contentType;
}
@Override public long contentLength() {
return file.length();
}
@Override public void writeTo(BufferedSink sink) throws IOException {
try (Source source = Okio.source(file)) {
sink.writeAll(source);
}
}
};
}Will be seen that the body holds the document, Content-Lengthreturn to the size of the file, the contents of all transferred to the Sourceobject.
This is indeed a point before very easy to overlook the few bodies to request for additional processing operation, and once this operation becomes a large one-time memory allocation, is likely to cause OOM. So how do we solve it? Signature method and ? how to deal with it turned out that the method signature here stole a lazy - it only reads the first 4K content of the incoming body, and then only encrypted for this part, as passed in the memory array itself and how much do not consider fully the risk and trouble threw external (excellent SDK!).
Fast way is to list the white list, not the interface for uploading validation server endorsement, but it is easy to exhaustive, and increased maintenance costs, people want to write a signature method sdk another suitable interface equal to their life, they still have to from the fundamental solution since the signature method reads only the first 4K content, we will only read part of the first 4K of memory sub-arrays can not be required as a method of so our aim is:? expectations RequestBodycan read part but not all of the contents . can inherit RequestBodyrewrite it writeTo? Yes, but unrealistic, impossible to replace all the existing RequestBodyimplementation class, but ok framework also possible to create private implementation class, so only for the writeToparameters of BufferedSinkfuss, come to understand BufferedSinkhow they are okhttp framework called.
BufferedSinkRelated classes include Buffer, Source, belong Okio frame, okhttp Okio only based on the cook, not Okio operation io java directly, but a separate write io operations, in particular operation of the data buffer. The above description then, Sourceis how to create, but it is also how to operate BufferedSinkin? Okio.javain:
public static Source source(File file) throws FileNotFoundException {
if (file == null) throw new IllegalArgumentException("file == null");
return source(new FileInputStream(file));
}
public static Source source(InputStream in) {
return source(in, new Timeout());
}
private static Source source(final InputStream in, final Timeout timeout) {
return new Source() {
@Override public long read(Buffer sink, long byteCount) throws IOException {
try {
timeout.throwIfReached();
Segment tail = sink.writableSegment(1);
int maxToCopy = (int) Math.min(byteCount, Segment.SIZE - tail.limit);
int bytesRead = in.read(tail.data, tail.limit, maxToCopy);
if (bytesRead == -1) return -1;
tail.limit += bytesRead;
sink.size += bytesRead;
return bytesRead;
} catch (AssertionError e) {
if (isAndroidGetsocknameError(e)) throw new IOException(e);
throw e;
}
}
@Override public void close() throws IOException {
in.close();
}
@Override public Timeout timeout() {
return timeout;
}
};
}SourceThe file as input stream inputstreamconducted various read, but its readmethod parameters, it was a Bufferexample, it is where he comes from, and how they BufferedSinkrelated? We had no choice but to continue to see BufferedSink.writeAllthe realization of the body.
BufferedSinkThe implementation class is Buffer, then its writeAllmethods:
@Override public long writeAll(Source source) throws IOException {
if (source == null) throw new IllegalArgumentException("source == null");
long totalBytesRead = 0;
for (long readCount; (readCount = source.read(this, Segment.SIZE)) != -1; ) {
totalBytesRead += readCount;
}
return totalBytesRead;
}The original is explicitly call the Source.read(Buffer,long)method, so that string up, that Bufferargument but it was their own.
Can be determined basically as long as the implement BufferedSinkinterface classes, and then determines the content of the read writing is stopped exceeds a specified size can meet the object is returned, it can be the name FixedSizeSink.
But the trouble is that BufferedSinkthe interface is very much, nearly 30 methods, the framework does not know which method will be called at any time to achieve only all! Followed by the interface method parameter okio has a lot of class, usage of these classes need to know, otherwise, if the wrong counterproductive. so understanding a class of turned into knowledge of more than one class, no way can only bite the bullet and write.
The first interface is a bit boring: Buffer buffer(); BufferedSinkreturns an Bufferinstance for external calls, BufferedSinkthe body that is achieved Buffer, then a return Buffer?! Looked a long guess BufferedSinkis to provide a buffer object can be written, but also lazy framework of the re decoupling the interface that set out the (alas, we are how simple how come). so FixedSizeSinkat least you need to hold an Bufferobject that make the actual data cache, while in need Source.read(Buffer ,long)as a parameter of the last places.
At the same time we can see RequestBodyan implementation class FormBody, using this Bufferwrite some data objects directly:
private long writeOrCountBytes(@Nullable BufferedSink sink, boolean countBytes) {
long byteCount = 0L;
Buffer buffer;
if (countBytes) {
buffer = new Buffer();
} else {
buffer = sink.buffer();
}
for (int i = 0, size = encodedNames.size(); i < size; i++) {
if (i > 0) buffer.writeByte('&');
buffer.writeUtf8(encodedNames.get(i));
buffer.writeByte('=');
buffer.writeUtf8(encodedValues.get(i));
}
if (countBytes) {
byteCount = buffer.size();
buffer.clear();
}
return byteCount;
}There are such operations is likely to limit the size of the buffer can not change! However, the amount of data should be relatively small and the usage scene is relatively small, we should be able to specify the size of the coverage of this situation.
Then there is an interface BufferedSink write(ByteString byteString), but also have to know ByteStringhow to use, really physically and mentally exhausted ah ...
@Override public Buffer write(ByteString byteString) {
byteString.write(this);
return this;
}BufferAchieve body where you can directly call ByteString.write(Buffer)because access is the package name, realize their own FixedSizeSinkdeclared in the same package name and package okio;can also be used as such, if the other package name can only be converted to byte[], and ByteStringshould not otherwise it would not be so engaged (not found ByteString a piece of data reading method):
@Override
public BufferedSink write(@NotNull ByteString byteString) throws IOException {
byte[] bytes = byteString.toByteArray();
this.write(bytes);
return this;
}Anyway, these object into a memory array or Buffercan accept arguments hold up!
The focus of concern writeAllbut relatively better realize that we continuously reads the contents of a specified length content length until we reached the threshold on the line.
And a boring point is read / write data flow direction of the various objects:
Caller.read(Callee)/Caller.write(Callee), the Callee to Caller from some, some contrast, is a subclass of the whole little headache ......
Finally, the complete code, if you find any potential problems can also exchange under ~:
public class FixedSizeSink implements BufferedSink {
private static final int SEGMENT_SIZE = 4096;
private final Buffer mBuffer = new Buffer();
private final int mLimitSize;
private FixedSizeSink(int size) {
this.mLimitSize = size;
}
@Override
public Buffer buffer() {
return mBuffer;
}
@Override
public BufferedSink write(@NotNull ByteString byteString) throws IOException {
byte[] bytes = byteString.toByteArray();
this.write(bytes);
return this;
}
@Override
public BufferedSink write(@NotNull byte[] source) throws IOException {
this.write(source, 0, source.length);
return this;
}
@Override
public BufferedSink write(@NotNull byte[] source, int offset,
int byteCount) throws IOException {
long available = mLimitSize - mBuffer.size();
int count = Math.min(byteCount, (int) available);
android.util.Log.d(TAG, String.format("FixedSizeSink.offset=%d,"
"count=%d,limit=%d,size=%d",
offset, byteCount, mLimitSize, mBuffer.size()));
if (count > 0) {
mBuffer.write(source, offset, count);
}
return this;
}
@Override
public long writeAll(@NotNull Source source) throws IOException {
this.write(source, mLimitSize);
return mBuffer.size();
}
@Override
public BufferedSink write(@NotNull Source source, long byteCount) throws IOException {
final long count = Math.min(byteCount, mLimitSize - mBuffer.size());
final long BUFFER_SIZE = Math.min(count, SEGMENT_SIZE);
android.util.Log.d(TAG, String.format("FixedSizeSink.count=%d,limit=%d"
",size=%d,segment=%d",
byteCount, mLimitSize, mBuffer.size(), BUFFER_SIZE));
long totalBytesRead = 0;
long readCount;
while (totalBytesRead < count && (readCount = source.read(mBuffer, BUFFER_SIZE)) != -1) {
totalBytesRead = readCount;
}
return this;
}
@Override
public int write(ByteBuffer src) throws IOException {
final int available = mLimitSize - (int) mBuffer.size();
if (available < src.remaining()) {
byte[] bytes = new byte[available];
src.get(bytes);
this.write(bytes);
return bytes.length;
} else {
return mBuffer.write(src);
}
}
@Override
public void write(@NotNull Buffer source, long byteCount) throws IOException {
mBuffer.write(source, Math.min(byteCount, mLimitSize - mBuffer.size()));
}
@Override
public BufferedSink writeUtf8(@NotNull String string) throws IOException {
mBuffer.writeUtf8(string);
return this;
}
@Override
public BufferedSink writeUtf8(@NotNull String string, int beginIndex, int endIndex)
throws IOException {
mBuffer.writeUtf8(string, beginIndex, endIndex);
return this;
}
@Override
public BufferedSink writeUtf8CodePoint(int codePoint) throws IOException {
mBuffer.writeUtf8CodePoint(codePoint);
return this;
}
@Override
public BufferedSink writeString(@NotNull String string,
@NotNull Charset charset) throws IOException {
mBuffer.writeString(string, charset);
return this;
}
@Override
public BufferedSink writeString(@NotNull String string, int beginIndex, int endIndex,
@NotNull Charset charset) throws IOException {
mBuffer.writeString(string, beginIndex, endIndex, charset);
return this;
}
@Override
public BufferedSink writeByte(int b) throws IOException {
mBuffer.writeByte(b);
return this;
}
@Override
public BufferedSink writeShort(int s) throws IOException {
mBuffer.writeShort(s);
return this;
}
@Override
public BufferedSink writeShortLe(int s) throws IOException {
mBuffer.writeShortLe(s);
return this;
}
@Override
public BufferedSink writeInt(int i) throws IOException {
mBuffer.writeInt(i);
return this;
}
@Override
public BufferedSink writeIntLe(int i) throws IOException {
mBuffer.writeIntLe(i);
return this;
}
@Override
public BufferedSink writeLong(long v) throws IOException {
mBuffer.writeLong(v);
return this;
}
@Override
public BufferedSink writeLongLe(long v) throws IOException {
mBuffer.writeLongLe(v);
return this;
}
@Override
public BufferedSink writeDecimalLong(long v) throws IOException {
mBuffer.writeDecimalLong(v);
return this;
}
@Override
public BufferedSink writeHexadecimalUnsignedLong(long v) throws IOException {
mBuffer.writeHexadecimalUnsignedLong(v);
return this;
}
@Override
public void flush() throws IOException {
mBuffer.flush();
}
@Override
public BufferedSink emit() throws IOException {
mBuffer.emit();
return this;
}
@Override
public BufferedSink emitCompleteSegments() throws IOException {
mBuffer.emitCompleteSegments();
return this;
}
@Override
public OutputStream outputStream() {
return mBuffer.outputStream();
}
@Override
public boolean isOpen() {
return mBuffer.isOpen();
}
@Override
public Timeout timeout() {
return mBuffer.timeout();
}
@Override
public void close() throws IOException {
mBuffer.close();
}
}