DiscUtils.Streams

Aligns I/O to a given block size.

Uses the read-modify-write pattern to align I/O.

Settings controlling BlockCache instances.

Initializes a new instance of the BlockCacheSettings class.

The cache settings.

Gets or sets the size (in bytes) of each cached block.

Gets or sets the maximum read size that will be cached.

Large reads are not cached, on the assumption they will not be repeated. This setting controls what is considered 'large'. Any read that is more than this many bytes will not be cached.

Gets or sets the optimum size of a read to the wrapped stream.

This value must be a multiple of BlockSize.

Gets or sets the size (in bytes) of the read cache.

Statistical information about the effectiveness of a BlockCache instance.

Gets the number of free blocks in the read cache.

Gets the number of requested 'large' reads, as defined by the LargeReadSize setting.

Gets the number of times a read request was serviced (in part or whole) from the cache.

Gets the number of time a read request was serviced (in part or whole) from the wrapped stream.

Gets the total number of requested reads.

Gets the total number of reads passed on by the cache.

Gets the total number of requested writes.

Gets the number of requested unaligned reads.

Unaligned reads are reads where the read doesn't start on a multiple of the block size.

Gets the number of requested unaligned writes.

Unaligned writes are writes where the write doesn't start on a multiple of the block size.

A stream implementing a block-oriented read cache.

Initializes a new instance of the BlockCacheStream class.

The stream to wrap. Whether to assume ownership of toWrap.

Initializes a new instance of the BlockCacheStream class.

The stream to wrap. Whether to assume ownership of toWrap. The cache settings.

Gets an indication as to whether the stream can be read.

Gets an indication as to whether the stream position can be changed.

Gets an indication as to whether the stream can be written to.

Gets the parts of the stream that are stored.

This may be an empty enumeration if all bytes are zero.

Gets the length of the stream.

Gets and sets the current stream position.

Gets the performance statistics for this instance.

Gets the parts of a stream that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Reads data from the stream.

The buffer to fill. The buffer offset to start from. The number of bytes to read. The number of bytes read.

Flushes the stream.

Moves the stream position.

The origin-relative location. The base location. The new absolute stream position.

Sets the length of the stream.

The new length.

Writes data to the stream at the current location.

The data to write. The first byte to write from buffer. The number of bytes to write.

Disposes of this instance, freeing up associated resources.

true if invoked from Dispose, else false.

Abstract base class for implementations of IBuffer.

Gets a value indicating whether this buffer can be read.

Gets a value indicating whether this buffer can be modified.

Gets the current capacity of the buffer, in bytes.

Gets the parts of the stream that are stored.

This may be an empty enumeration if all bytes are zero.

Reads from the buffer into a byte array.

The offset within the buffer to start reading. The destination byte array. The start offset within the destination buffer. The number of bytes to read. The actual number of bytes read.

Writes a byte array into the buffer.

The start offset within the buffer. The source byte array. The start offset within the source byte array. The number of bytes to write.

Clears bytes from the buffer.

The start offset within the buffer. The number of bytes to clear. Logically equivalent to writing count null/zero bytes to the buffer, some implementations determine that some (or all) of the range indicated is not actually stored. There is no direct, automatic, correspondence to clearing bytes and them not being represented as an 'extent' - for example, the implementation of the underlying stream may not permit fine-grained extent storage. It is always safe to call this method to 'zero-out' a section of a buffer, regardless of the underlying buffer implementation.

Flushes all data to the underlying storage.

The default behaviour, implemented by this class, is to take no action.

Sets the capacity of the buffer, truncating if appropriate.

The desired capacity of the buffer.

Gets the parts of a buffer that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Converts a Buffer into a Stream.

Initializes a new instance of the BufferStream class.

The buffer to use. The access permitted to clients.

Gets an indication of whether read access is permitted.

Gets an indication of whether seeking is permitted.

Gets an indication of whether write access is permitted.

Gets the stored extents within the sparse stream.

Gets the length of the stream (the capacity of the underlying buffer).

Gets and sets the current position within the stream.

Flushes all data to the underlying storage.

Reads a number of bytes from the stream.

The destination buffer. The start offset within the destination buffer. The number of bytes to read. The number of bytes read.

Changes the current stream position.

The origin-relative stream position. The origin for the stream position. The new stream position.

Sets the length of the stream (the underlying buffer's capacity).

The new length of the stream.

Writes a buffer to the stream.

The buffer to write. The starting offset within buffer. The number of bytes to write.

Clears bytes from the stream.

The number of bytes (from the current position) to clear. Logically equivalent to writing count null/zero bytes to the stream, some implementations determine that some (or all) of the range indicated is not actually stored. There is no direct, automatic, correspondence to clearing bytes and them not being represented as an 'extent' - for example, the implementation of the underlying stream may not permit fine-grained extent storage. It is always safe to call this method to 'zero-out' a section of a stream, regardless of the underlying stream implementation.

Gets the parts of a stream that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Interface shared by all buffers.

Buffers are very similar to streams, except the buffer has no notion of 'current position'. All I/O operations instead specify the position, as needed. Buffers also support sparse behaviour.

Gets a value indicating whether this buffer can be read.

Gets a value indicating whether this buffer can be modified.

Gets the current capacity of the buffer, in bytes.

Gets the parts of the buffer that are stored.

This may be an empty enumeration if all bytes are zero.

Reads from the buffer into a byte array.

The offset within the buffer to start reading. The destination byte array. The start offset within the destination buffer. The number of bytes to read. The actual number of bytes read.

Writes a byte array into the buffer.

The start offset within the buffer. The source byte array. The start offset within the source byte array. The number of bytes to write.

Clears bytes from the buffer.

Flushes all data to the underlying storage.

Sets the capacity of the buffer, truncating if appropriate.

The desired capacity of the buffer.

Gets the parts of a buffer that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Class representing a portion of an existing buffer.

Initializes a new instance of the SubBuffer class.

The parent buffer. The first byte in represented by this sub-buffer. The number of bytes of represented by this sub-buffer.

Can this buffer be read.

Can this buffer be modified.

Gets the current capacity of the buffer, in bytes.

Gets the parts of the buffer that are stored.

This may be an empty enumeration if all bytes are zero.

Flushes all data to the underlying storage.

Reads from the buffer into a byte array.

The offset within the buffer to start reading. The destination byte array. The start offset within the destination buffer. The number of bytes to read. The actual number of bytes read.

Writes a byte array into the buffer.

The start offset within the buffer. The source byte array. The start offset within the source byte array. The number of bytes to write.

Sets the capacity of the buffer, truncating if appropriate.

The desired capacity of the buffer.

Gets the parts of a buffer that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Gets the parts of the stream that are stored.

This may be an empty enumeration if all bytes are zero.

Base class for objects that can dynamically construct a stream.

Builds a new stream.

The stream created by the StreamBuilder instance.

Writes the stream contents to an existing stream.

The stream to write to.

Writes the stream contents to a file.

The file to write to.

Represents a stream that is circular, so reads and writes off the end of the stream wrap.

The concatenation of multiple streams (read-only, for now).

Common interface for reading structures to/from byte arrays.

Gets the total number of bytes the structure occupies.

Reads the structure from a byte array.

The buffer to read from. The buffer offset to start reading from. The number of bytes read.

Writes a structure to a byte array.

The buffer to write to. The buffer offset to start writing at.

Represents a stream with a specified length

since the wrapped stream may not support there is no validation of the specified length

Base class for streams that are essentially a mapping onto a parent stream.

This class provides access to the mapping underlying the stream, enabling callers to convert a byte range in this stream into one or more ranges in the parent stream.

Converts any stream into a non-linear stream.

The stream to convert. true to have the new stream dispose the wrapped stream when it is disposed. A sparse stream. The wrapped stream is assumed to be a linear stream (such that any byte range maps directly onto the parent stream).

Converts any stream into a non-linear stream.

The stream to convert. true to have the new stream dispose the wrapped stream when it is disposed. The set of extents actually stored in stream. A sparse stream. The wrapped stream is assumed to be a linear stream (such that any byte range maps directly onto the parent stream).

Maps a logical range down to storage locations.

The first logical range to map. The length of the range to map. One or more stream extents specifying the storage locations that correspond to the identified logical extent range. As far as possible, the stream extents are returned in logical disk order - however, due to the nature of non-linear streams, not all of the range may actually be stored, or some or all of the range may be compressed - thus reading the returned stream extents is not equivalent to reading the logical disk range.

Stream wrapper to allow forward only seeking on not seekable streams

Event arguments indicating progress on pumping a stream.

Gets or sets the number of bytes read from InputStream.

Gets or sets the number of bytes written to OutputStream.

Gets or sets the absolute position in OutputStream.

Gets or sets the absolute position in InputStream.

Base class for reading binary data from a stream.

Class for reading little-endian data from a stream.

A wrapper stream that enables you to take a snapshot, pushing changes into a side buffer.

Once a snapshot is taken, you can discard subsequent changes or merge them back into the wrapped stream.

Records which byte ranges in diffStream hold changes.

Can't use _diffStream's own tracking because that's based on it's internal block size, not on the _actual_ bytes stored.

Captures changes to the base stream (when enabled).

Indicates that no writes should be permitted.

The saved stream position (if the diffStream is active).

Initializes a new instance of the SnapshotStream class.

The stream to wrap. Indicates if this stream should control the lifetime of baseStream.

Gets an indication as to whether the stream can be read.

Gets an indication as to whether the stream position can be changed.

Gets an indication as to whether the stream can be written to.

This property is orthogonal to Freezing/Thawing, it's perfectly possible for a stream to be frozen and this method return true.

Returns an enumeration over the parts of the stream that contain real data.

Gets the length of the stream.

Gets and sets the current stream position.

Prevents any write operations to the stream.

Useful to prevent changes whilst inspecting the stream.

Re-permits write operations to the stream.

Takes a snapshot of the current stream contents.

Reverts to a previous snapshot, discarding any changes made to the stream.

Discards the snapshot any changes made after the snapshot was taken are kept.

Flushes the stream.

Reads data from the stream.

The buffer to fill. The buffer offset to start from. The number of bytes to read. The number of bytes read.

Moves the stream position.

The origin-relative location. The base location. The new absolute stream position.

Sets the length of the stream.

The new length.

Writes data to the stream at the current location.

The data to write. The first byte to write from buffer. The number of bytes to write.

Disposes of this instance.

true if called from Dispose(), else false.

A sparse in-memory buffer.

This class is useful for storing large sparse buffers in memory, unused chunks of the buffer are not stored (assumed to be zero).

Initializes a new instance of the SparseMemoryBuffer class.

The size of each allocation chunk.

Gets the (sorted) list of allocated chunks, as chunk indexes.

An enumeration of chunk indexes. This method returns chunks as an index rather than absolute stream position. For example, if ChunkSize is 16KB, and the first 32KB of the buffer is actually stored, this method will return 0 and 1. This indicates the first and second chunks are stored.

Indicates this stream can be read (always true).

Indicates this stream can be written (always true).

Gets the current capacity of the sparse buffer (number of logical bytes stored).

Gets the size of each allocation chunk.

Accesses this memory buffer as an infinite byte array.

The buffer position to read. The byte stored at this position (or Zero if not explicitly stored).

Reads a section of the sparse buffer into a byte array.

The offset within the sparse buffer to start reading. The destination byte array. The start offset within the destination buffer. The number of bytes to read. The actual number of bytes read.

Writes a byte array into the sparse buffer.

The start offset within the sparse buffer. The source byte array. The start offset within the source byte array. The number of bytes to write.

Clears bytes from the buffer.

The start offset within the buffer. The number of bytes to clear.

Sets the capacity of the sparse buffer, truncating if appropriate.

The desired capacity of the buffer. This method does not allocate any chunks, it merely records the logical capacity of the sparse buffer. Writes beyond the specified capacity will increase the capacity.

Gets the parts of a buffer that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Provides a sparse equivalent to MemoryStream.

Initializes a new instance of the SparseMemoryStream class.

The created instance permits read and write access.

Initializes a new instance of the SparseMemoryStream class.

The buffer to use. The access permitted to clients.

Represents a sparse stream.

A sparse stream is a logically contiguous stream where some parts of the stream aren't stored. The unstored parts are implicitly zero-byte ranges.

Gets the parts of the stream that are stored.

This may be an empty enumeration if all bytes are zero.

Converts any stream into a sparse stream.

The stream to convert. true to have the new stream dispose the wrapped stream when it is disposed. A sparse stream. The returned stream has the entire wrapped stream as a single extent.

Converts any stream into a sparse stream.

The stream to convert. true to have the new stream dispose the wrapped stream when it is disposed. The set of extents actually stored in stream. A sparse stream. The returned stream has the entire wrapped stream as a single extent.

Efficiently pumps data from a sparse stream to another stream.

The sparse stream to pump from. The stream to pump to. must support seeking.

Efficiently pumps data from a sparse stream to another stream.

The stream to pump from. The stream to pump to. The smallest sequence of zero bytes that will be skipped when writing to . must support seeking.

Wraps a sparse stream in a read-only wrapper, preventing modification.

The stream to make read-only. Whether to transfer responsibility for calling Dispose on toWrap. The read-only stream.

Clears bytes from the stream.

Gets the parts of a stream that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Converts a Stream into an IBuffer instance.

Initializes a new instance of the StreamBuffer class.

The stream to wrap. Whether to dispose stream, when this object is disposed.

Can this buffer be read.

Can this buffer be written.

Gets the current capacity of the buffer, in bytes.

Gets the parts of the stream that are stored.

This may be an empty enumeration if all bytes are zero.

Disposes of this instance.

Reads from the buffer into a byte array.

The offset within the buffer to start reading. The destination byte array. The start offset within the destination buffer. The number of bytes to read. The actual number of bytes read.

Writes a byte array into the buffer.

The start offset within the buffer. The source byte array. The start offset within the source byte array. The number of bytes to write.

Flushes all data to the underlying storage.

Sets the capacity of the buffer, truncating if appropriate.

The desired capacity of the buffer.

Gets the parts of a buffer that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Represents a range of bytes in a stream.

This is normally used to represent regions of a SparseStream that are actually stored in the underlying storage medium (rather than implied zero bytes). Extents are stored as a zero-based byte offset (from the beginning of the stream), and a byte length.

Initializes a new instance of the StreamExtent class.

The start of the extent. The length of the extent.

Gets the start of the extent (in bytes).

Compares this stream extent to another.

The extent to compare. Value greater than zero if this extent starts after other, zero if they start at the same position, else a value less than zero.

Indicates if this StreamExtent is equal to another.

The extent to compare. true if the extents are equal, else false.

Calculates the union of a list of extents with another extent.

The list of extents. The other extent. The union of the extents.

Calculates the union of the extents of multiple streams.

The stream extents. The union of the extents from multiple streams. A typical use of this method is to calculate the combined set of stored extents from a number of overlayed sparse streams.

Calculates the intersection of the extents of a stream with another extent.

The stream extents. The extent to intersect. The intersection of the extents.

Calculates the intersection of the extents of multiple streams.

The stream extents. The intersection of the extents from multiple streams. A typical use of this method is to calculate the extents in a region of a stream..

Calculates the subtraction of the extents of a stream by another extent.

The stream extents. The extent to subtract. The subtraction of other from extents.

Calculates the subtraction of the extents of a stream by another stream.

The stream extents to subtract from. The stream extents to subtract. The subtraction of the extents of b from a.

Calculates the inverse of the extents of a stream.

The stream extents to inverse. The inverted extents. This method assumes a logical stream addressable from 0 to long.MaxValue, and is undefined should any stream extent start at less than 0. To constrain the extents to a specific range, use the Intersect method.

Offsets the extents of a stream.

The stream extents. The amount to offset the extents by. The stream extents, offset by delta.

Returns the number of blocks containing stream data.

The stream extents. The size of each block. The number of blocks containing stream data. This method logically divides the stream into blocks of a specified size, then indicates how many of those blocks contain actual stream data.

Returns all of the blocks containing stream data.

The stream extents. The size of each block. Ranges of blocks, as block indexes. This method logically divides the stream into blocks of a specified size, then indicates ranges of blocks that contain stream data.

The equality operator.

The first extent to compare. The second extent to compare. Whether the two extents are equal.

The inequality operator.

The first extent to compare. The second extent to compare. Whether the two extents are different.

The less-than operator.

The first extent to compare. The second extent to compare. Whether a is less than b.

The greater-than operator.

The first extent to compare. The second extent to compare. Whether a is greater than b.

Returns a string representation of the extent as [start:+length].

The string representation.

Indicates if this stream extent is equal to another object.

The object to test. true if obj is equivalent, else false.

Gets a hash code for this extent.

The extent's hash code.

Utility class for pumping the contents of one stream into another.

This class is aware of sparse streams, and will avoid copying data that is not valid in the source stream. This functionality should normally only be used when the destination stream is known not to contain any existing data.

Initializes a new instance of the StreamPump class.

The stream to read from. The stream to write to. The size of each sparse chunk.

Gets or sets the amount of data to read at a time from InputStream.

Gets the number of bytes read from InputStream.

Gets the number of bytes written to OutputStream.

Gets or sets the stream that will be read from.

Gets or sets the stream that will be written to.

Gets or sets, for sparse transfers, the size of each chunk.

A chunk is transfered if any byte in the chunk is valid, otherwise it is not. This value should normally be set to reflect the underlying storage granularity of OutputStream.

Gets or sets a value indicating whether to enable the sparse copy behaviour (default true).

Event raised periodically through the pump operation.

This event is signalled synchronously, so to avoid slowing the pumping activity implementations should return quickly.

Performs the pump activity, blocking until complete.

Provides a thread-safe wrapping around a sparse stream.

Streams are inherently not thread-safe (because read/write is not atomic w.r.t. Position). This method enables multiple 'views' of a stream to be created (each with their own Position), and ensures only a single operation is executing on the wrapped stream at any time. This example shows the pattern of use:


            SparseStream baseStream = ...;
            ThreadSafeStream tss = new ThreadSafeStream(baseStream);
            for(int i = 0; i < 10; ++i)
            {
              SparseStream streamForThread = tss.OpenView();
            }

This results in 11 streams that can be used in different streams - tss and ten 'views' created from tss. Note, the stream length cannot be changed.

Initializes a new instance of the ThreadSafeStream class.

The stream to wrap. Do not directly modify toWrap after wrapping it, unless the thread-safe views will no longer be used.

Initializes a new instance of the ThreadSafeStream class.

The stream to wrap. Whether to transfer ownership of toWrap to the new instance. Do not directly modify toWrap after wrapping it, unless the thread-safe views will no longer be used.

Gets a value indicating if this stream supports reads.

Gets a value indicating if this stream supports seeking (always true).

Gets a value indicating if this stream supports writes (currently, always false).

Gets the parts of the stream that are stored.

This may be an empty enumeration if all bytes are zero.

Gets the length of the stream.

Gets the current stream position - each 'view' has it's own Position.

Opens a new thread-safe view on the stream.

The new view.

Gets the parts of a stream that are stored, within a specified range.

The offset of the first byte of interest. The number of bytes of interest. An enumeration of stream extents, indicating stored bytes.

Causes the stream to flush all changes.

Reads data from the stream.

The buffer to fill. The first byte in buffer to fill. The requested number of bytes to read. The actual number of bytes read.

Changes the current stream position (each view has it's own Position).

The relative location to move to. The origin of the location. The new location as an absolute position.

Sets the length of the stream (not supported).

The new length.

Writes data to the stream (not currently supported).

The data to write. The first byte to write. The number of bytes to write.

Disposes of this instance, invalidating any remaining views.

true if disposing, lese false.

Helper to count the number of bits set in a byte or byte[]

count the number of bits set in

the number of bits set in

count the number of bits set in each entry of

the to process the values offset to start from the number of bytes to count

Primitive conversion from Unicode to ASCII that preserves special characters.

The string to convert. The buffer to fill. The start of the string in the buffer. The number of characters to convert. The built-in ASCIIEncoding converts characters of codepoint > 127 to ?, this preserves those code points by removing the top 16 bits of each character.

Primitive conversion from ASCII to Unicode that preserves special characters.

The data to convert. The first byte to convert. The number of bytes to convert. The string. The built-in ASCIIEncoding converts characters of codepoint > 127 to ?, this preserves those code points.

Primitive conversion from ASCII to Unicode that stops at a null-terminator.

The data to convert. The first byte to convert. The number of bytes to convert. The string. The built-in ASCIIEncoding converts characters of codepoint > 127 to ?, this preserves those code points.

Round up a value to a multiple of a unit size.

The value to round up. The unit (the returned value will be a multiple of this number). The rounded-up value.

Round up a value to a multiple of a unit size.

The value to round up. The unit (the returned value will be a multiple of this number). The rounded-up value.

Round down a value to a multiple of a unit size.

The value to round down. The unit (the returned value will be a multiple of this number). The rounded-down value.

Calculates the CEIL function.

The value to divide. The value to divide by. The value of CEIL(numerator/denominator).

Calculates the CEIL function.

The value to divide. The value to divide by. The value of CEIL(numerator/denominator).

Calculates the CEIL function.

The value to divide. The value to divide by. The value of CEIL(numerator/denominator).

Enumeration used to indicate transfer of disposable objects.

Indicates there is no transfer of ownership.

Indicates ownership of the stream is transfered, the owner should dispose of the stream when appropriate.

Represents a range of values.

The type of the offset element. The type of the size element.

Initializes a new instance of the Range class.

The offset (i.e. start) of the range. The size of the range.

Gets the size of the range.

Gets the offset (i.e. start) of the range.

Compares this range to another.

The range to compare. true if the ranges are equivalent, else false.

Merges sets of ranges into chunks.

The ranges to merge. The size of each chunk. Ranges combined into larger chunks. The type of the offset and count in the ranges.

Returns a string representation of the extent as [start:+length].

The string representation.

Validates standard buffer, offset, count parameters to a method.

The byte array to read from / write to. The starting offset in buffer. The number of bytes to read / write.

Read bytes until buffer filled or throw EndOfStreamException.

The stream to read. The buffer to populate. Offset in the buffer to start. The number of bytes to read.

Read bytes until buffer filled or throw EndOfStreamException.

The stream to read. The number of bytes to read. The data read from the stream.

Read bytes until buffer filled or throw EndOfStreamException.

The stream to read. The position in buffer to read from. The buffer to populate. Offset in the buffer to start. The number of bytes to read.

Read bytes until buffer filled or throw EndOfStreamException.

The buffer to read. The position in buffer to read from. The number of bytes to read. The data read from the stream.

Read bytes until buffer filled or EOF.

The stream to read. The buffer to populate. Offset in the buffer to start. The number of bytes to read. The number of bytes actually read.

Read bytes until buffer filled or EOF.

The stream to read. The position in buffer to read from. The buffer to populate. Offset in the buffer to start. The number of bytes to read. The number of bytes actually read.

Read bytes until buffer filled or throw EndOfStreamException.

The buffer to read. The data read from the stream.

Reads a disk sector (512 bytes).

The stream to read. The sector data as a byte array.

Reads a structure from a stream.

The type of the structure. The stream to read. The structure.

Reads a structure from a stream.

The type of the structure. The stream to read. The number of bytes to read. The structure.

Writes a structure to a stream.

The type of the structure. The stream to write to. The structure to write.

Copies the contents of one stream to another.

The stream to copy from. The destination stream. Copying starts at the current stream positions.

Base class for streams that wrap another stream.

The type of stream to wrap. Provides the default implementation of methods & properties, so wrapping streams need only override the methods they need to intercept.

Base class for streams that wrap another stream.

Provides the default implementation of methods & properties, so wrapping streams need only override the methods they need to intercept.

A stream that returns Zero's.