cppadvanced60 minutes

Implement a Multi-threaded File Search Utility in C++

Create an advanced multi-threaded file search utility in C++ that scans directories recursively to find files containing a specific keyword. The program should optimize search performance using concurrency and handle large directory trees efficiently.

Challenge prompt

Write a C++ program that takes two arguments: a directory path and a search keyword. The program should recursively search all files in the directory and its subdirectories for the keyword. Implement multithreading to parallelize file processing for improved performance. Your output should include the full path of each file containing the keyword and the line numbers where the keyword appears. Ensure thread safety and efficient resource management.

Guidance

• Use C++17 filesystem library for recursive directory traversal.
• Deploy a thread pool or use std::async to manage multiple threads efficiently.
• Implement synchronization mechanisms to safely write results concurrently.
• Optimize I/O operations to handle potentially large files with minimal bottlenecks.

Hints

• Consider buffering file content and avoid loading entire huge files into memory.
• Use mutexes or thread-safe queues to collect matching results from threads.
• Limit the number of active threads based on hardware concurrency.

Starter code

#include <iostream>
#include <filesystem>
#include <fstream>
#include <string>
#include <vector>
#include <thread>
#include <mutex>
#include <future>

namespace fs = std::filesystem;

std::mutex output_mutex;

void searchFile(const fs::path &filePath, const std::string &keyword) {
    std::ifstream file(filePath);
    if (!file.is_open()) return;

    std::string line;
    int lineNumber = 0;
    std::vector<int> matchedLines;

    while (std::getline(file, line)) {
        ++lineNumber;
        if (line.find(keyword) != std::string::npos) {
            matchedLines.push_back(lineNumber);
        }
    }

    if (!matchedLines.empty()) {
        std::lock_guard<std::mutex> lock(output_mutex);
        std::cout << "Found in: " << filePath << " Lines: ";
        for (int num : matchedLines) {
            std::cout << num << " ";
        }
        std::cout << std::endl;
    }
}

int main(int argc, char* argv[]) {
    if (argc != 3) {
        std::cerr << "Usage: " << argv[0] << " <directory_path> <keyword>" << std::endl;
        return 1;
    }

    const fs::path dirPath = argv[1];
    const std::string keyword = argv[2];

    // Your implementation here

    return 0;
}

Expected output

Found in: /path/to/file.txt Lines: 3 15 27 Found in: /path/to/subdir/log.txt Lines: 1 2 5

Core concepts

multithreadingfilesystem traversalfile I/Osynchronization

Challenge a Friend

Send this duel to someone else and see if they can solve it.