more about - System.arraycopy(...) equivalents for ByteBuffer/FileChannel (original) (raw)

Alan Bateman Alan.Bateman at oracle.com
Fri Nov 2 03:24:24 PDT 2012

On 02/11/2012 01:29, Jeff Hain wrote:

:

That's modulo the fact that MappedByteBuffers can't be explicitly and portably unmapped (Bug ID: 4724038), so you can't always use them. Yes, a very difficult issue that many people have looked at but didn't come up a solution that addresses all the concerns, see:

http://bugs.sun.com/view_bug.do?bug_id=4724038

That's also modulo the fact that, for BB to FC copies, if the FileChannel is not readable, you can't use MBB (no WRITEONLY mapping mode). A FileChannel.isReadable() method would allow to use MBB for readable (and writable) channels, without risk of an exception being thrown. It would although if code is running into then it suggests that it might not be using the API as intended.

:

That's also modulo the fact that when you copy between a MappedByteBuffer and a FileChannel, src and dst might share memory, but there is no API to figure out if and how, thus you don't know if you have to copy forward or backward (to avoid erasing bytes to copy with copied bytes). Right, there isn't anything in this API to know when buffers or mappings overlap.

:

I found FileChannel.write(ByteBuffer,long) to be very slow on WinXP/7, by a factor 15, when the ByteBuffer is "large" (like 500Ko), even if it is direct (mapped or not), in which case it resolves to sun.nio.ch.IOUtils.writeFromNativeBuffer (which I supposed to always be fast)). On Linux there was no slow down. This is another long standing issue, see:

http://bugs.sun.com/view_bug.do?bug_id=6265734

It comes down to Windows not having the equivalent of a pread/pwrite that doesn't change the global position. There are ideas on how to solve this but it requires significant refactoring, hopefully some day.

:

Intensive benches involving MBBs (namely FC to heap BB copies) were hanging from time to time, up to nearly a second, and then resumed at usual speed (slightly faster than temporary direct ByteBuffer approaches). ===> As a result, I completely disabled MBBs usage for my copies. Was this Windows 32-bit? We've seen a lot of issues with memory management on Windows where it takes time to write-out dirty pages and unmap the memory. I don't think we can do anything about this, assuming this is what you are running into. Windows 64-bit appears to be better although it can be a problem too when memory is over committed.

3) Non-performance remarks: When copying between direct ByteBuffers, the efficient way is to use ByteBuffer.put(ByteBuffer), but its spec says that it's about equivalent to "while (src.hasRemaining()) dst.put(src.get());", i.e. that it is not suited if memory is shared (which no API allows to figure out) and srcPos < dstPos (as raw memmory positions), since then it could erase bytes to copy with copied bytes. We could potentially clarify the spec here although warning about overlapping regions may require changes in several other places too.

:

FileChannelImpl.truncate(long): The bug we already talked about (the early return if size > size(), which masks the writability check, and the fact that size >= size() could be done instead) prevents reworking position whether truncation occurs or not, as the spec says (even if size > size(), position should be set to size if it is superior, but it isn't - or is it a spec bug?). This is a corner case which is probably why it was never reported before. We have a bug for it since you brought it up:

http://bugs.sun.com/view_bug.do?bug_id=8000330

FileChannelImpl.map(...): If mode is null, and assertions enabled, "assert (imode >= 0)" fails (I have somewhere in my head the idea that in JDK assertions should only check private code - well in a sense here it does :). If mode is passed as null then NullPointerException should be thrown. Another 10+ year issue that hasn't been noticed. I'll create a bug for this - thanks!

FileChannelImpl.transferFrom(...): This method can grow destination channel, but if the specified position (in dst) is > dst.size(), it just returns 0. It looks like a bug, as the spec says nothing about this surprising behavior. This is specified: " If the given position is greater than the file's current size then no bytes are transferred"

FileChannel.write(ByteBuffer,long): The Javadoc says that position is not updated, but if the channel is in append mode it might be (since then we have position = size, and this method can grow the file). There is a long standing bug tracking updating the spec on this issue, see:

http://bugs.sun.com/view_bug.do?bug_id=6924219

Thanks for bug reports, you've gone in corner cases that our tests for this area clearly haven't gone into.

-Alan

-------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.openjdk.java.net/pipermail/nio-dev/attachments/20121102/47ccd343/attachment-0001.html

More information about the nio-dev mailing list