Buffered streams

What are buffered streams

Streams are interfaces used to input and output from/to a device. For performance reasons, they might use buffers (A buffer is an array of data). In that case, output/input operations on a stream does not always directly write/read data to/from the device. If we execute an output operation on a buffered stream, it copies the data to its buffer and writes the data to the device only when the buffer is full or when the stream is flushed (When the stream is closed, it is flushed).

Why using buffered streams?

If we execute many small output operations (Write a few characters (bytes)) directly to a device like the console, it is not too bad performance-wise. However, if we do that to a storage device like a hard drive, this is very problematic. Storage devices are divided into small sections called sectors. The thing is that there is a minimum amount of data we can output/input to/from a storage device in one write operation. We can not write/read individual bytes, we must output/read whole sectors. A hard drive sector is usually made of at least 512 bytes. So if we directly execute 100 write operations of, for example, the string "Hello" (Which weight a total of 500 bytes) on a hard drive, we will actually output 51200 bytes (100 write operations of sectors of 512 bytes). If we had instead copied the string "Hello" 100 times to a buffer and then outputted the buffer to the hard drive, we would have only written 1 sector, so 512 bytes, to the device.

Buffering types

There is 2 types of buffering (Apart from no buffering): full buffering and line buffering. In both cases, output operations fill the buffer of the stream and when the buffer is either full of flushed, its content is written to the device linked to the stream. The difference is that with line buffering, each time the newline character is outputted, the buffer content is written (flushed) to the device.

The function fopen returns a fully buffered stream if it opens a storage device, but it is possible to change that. We must, however, change the buffering type before executing any operation on the stream.

By default, the size (in bytes) of the buffers used by buffered stream is equal to the macro BUFSIZ. It is possible, however, change the size of the buffer of a buffered stream.

The function setbuf

The function which header is void setbuf(FILE* stream, char* buffer), is used to assign the buffer received as second argument to the stream pointed by the first argument. Remember, a buffer is an array of data (bytes). The buffer must be of the size represented by the value of the macro BUFSIZ. If the second argument is a null pointer (The address 0), then the stream becomes non-buffered stream. Note that streams returned by the function fopen already have a buffer if they are buffered.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#include <stdio.h> #include <stdlib.h> int main() { FILE * stream = fopen("test.txt", "w"); // The stream 'stream' is fully buffered. setbuf(stream, 0); /* The stream 'stream' is now unbuffered. */ char *buffer = (char*)malloc(BUFSIZ); setbuf(stream, buffer); /* The stream 'stream' is now fully buffered again and it uses the buffer 'buffer'. */ fclose(stream); free(buffer); return 0; }

The function setvbuf

The function which header is int setvbuf(FILE* stream, char* buffer, int mode, size_t size) is used to define the buffering type of a stream and assign a buffer (Of a size we choose) to it. The first argument is a pointer to the stream, the second one is a pointer to the buffer, the third one is the buffering type of the stream and the last one is the size (In bytes) of the buffer. The third argument must be either the macro _IOFBF, for full buffering, _IOLBF, for line buffering and _IONBF for no buffering.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#include <stdio.h> #include <stdlib.h> int main() { FILE * stream = fopen("test.txt", "w"); // The stream 'stream' is fully buffered. char *buffer = (char*)malloc(5 * 1000 * 1000); // 5 MB setvbuf(stream, buffer, _IOLBF, 5000000); /* The stream 'stream' is now line buffered and it uses the buffer 'buffer', which has a size of 5 MB. */ fclose(stream); free(buffer); return 0; }

The function fflush

The function which header is int fflush(FILE* stream) is used to flush (Write the content of the buffer of a stream to the device it is linked to) the stream pointed by the stream object pointer given as argument. It returns 0 if the operation has been successfully executed. If is receives a null pointer as argument (The address 0), it flushes all the streams.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#include <stdio.h> int main() { FILE * stream = fopen("test.txt", "w"); /* Let us pretend there are output operations executed on the stream 'stream', here. */ fflush(stream); // Now, the stream 'stream' is flushed. // Let us pretend there are output operations executed on the stream 'stream', here. fclose(stream); // The function fclose also flushes the stream. return 0; }

Note on read/write streams

Buffered streams that are opened for reading and writing must be flushed after output operations had been executed, before we can read from them. Note that changing the position of the cursor inside a stream also flushes the stream.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#include <stdio.h> int main() { FILE * file = fopen("test.bin", "w+"); char array[] = {1, 2, 3, 4, 5}; char array2[5]; // Writes 5 bytes to the stream 'file' from the static array 'array'. fwrite(array, 1, 5, file); /* We set the cursor back to the beginning of the stream 'file'. Changing the position of the cursor inside a stream also has the effect of flushing it. We therefore do not need to use the function fflush here. */ fseek(file, 0, SEEK_SET); // We read 5 bytes from the stream 'file' into the array 'array2'. fread(array2, 1, 5, file); fclose(file); return 0; }

Here is an other example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
#include <stdio.h> int main() { FILE * file = fopen("test.bin", "w+"); char array[] = {1, 2, 3, 4, 5}; char array2[5]; // Write 5 bytes to the stream 'file' from the array 'array'. fwrite(array, 1, 5, file); /* We need to flush the stream before reading from it, because we outputted to it and the stream is buffered. */ fflush(file); // We read 5 bytes from the stream 'file' into the array 'array2'. fread(array2, 1, 5, file); fclose(file); return 0; }