C++: async file IO

Programming
, , , ,
1

Question

I have a super-fast M.2 drive, but my application is not able to utilize this speed when writing and reading large amounts of data. Is it possible to have truly asynchronous file IO in C++ to take full advantage of the drive’s advertised gigabytes per second? If so, how can this be done in a cross-platform way? Alternatively, can you recommend a library that is suitable for this task?

Below is a code snippet that illustrates how I am doing file IO in my program.

#include <fstream>
#include <thread>
#include <memory>
#include <string>

#include <Windows.h> // for SetThreadAffinityMask()

void stress_write(unsigned bytes, int num)
{
    std::ofstream out("temp" + std::to_string(num));
    for (unsigned i = 0; i < bytes; ++i)
    {
        out << char(i);
    }
}

void lock_thread(unsigned core_idx)
{
    SetThreadAffinityMask(GetCurrentThread(), 1LL << core_idx);
}

int main()
{
    std::ios_base::sync_with_stdio(false);
    lock_thread(0);

    auto worker_count = std::thread::hardware_concurrency() - 1;

    std::unique_ptr<std::thread[]> threads = std::make_unique<std::thread[]>(worker_count); // faster than std::vector

    for (int i = 0; i < worker_count; ++i)
    {
        threads[i] = std::thread(
            [](unsigned idx) {
                lock_thread(idx);
                stress_write(1'000'000'000, idx);
            },
            i + 1
        );
    }
    stress_write(1'000'000'000, 0);

    for (int i = 0; i < worker_count; ++i)
    {
        threads[i].join();
    }
}

My current implementation uses 100% CPU, but only 7-9% disk (around 190MB/s). Is there a way to increase this speed?

2

Yes, it is possible to have truly asynchronous file IO in C++. One cross-platform library that can be used for this task is Boost.Asio. With Boost.Asio, you can use asynchronous file IO operations to take full advantage of the drive’s advertised gigabytes per second. Below is an example of how to use Boost.Asio for asynchronous file IO:

#include <iostream>
#include <fstream>
#include <string>
#include <boost/asio.hpp>

int main()
{
    boost::asio::io_service io_service;
    boost::asio::steady_timer timer(io_service);
    std::ofstream file("test.txt");

    const std::size_t buffer_size = 1024 * 1024 * 1024;
    char* buffer = new char[buffer_size];

    // Start the timer to measure the time it takes to write to the file
    timer.expires_from_now(std::chrono::seconds(0));

    timer.async_wait([&](const boost::system::error_code& ec) {
        if (!ec) {
            auto start_time = std::chrono::steady_clock::now();

            // Write to the file asynchronously
            boost::asio::async_write(file, boost::asio::buffer(buffer, buffer_size),
                [&](const boost::system::error_code& ec, std::size_t bytes_transferred) {
                    auto end_time = std::chrono::steady_clock::now();
                    std::chrono::duration<double> elapsed_seconds = end_time - start_time;
                    std::cout << "Time elapsed: " << elapsed_seconds.count() << " seconds\n";
                });
        }
    });

    io_service.run();

    delete[] buffer;
    return 0;
}

This code snippet writes 1GB of data to a file asynchronously using Boost.Asio. The async_write function writes the buffer to the file in a non-blocking fashion, allowing other operations to be performed while the write is in progress. The async_wait function starts a timer to measure the time it takes to write to the file, and the io_service.run() function blocks until all asynchronous operations have completed.