[kernel] Move Execution of Side Effect Kernel Calls to Dedicated Kernel Thread

[ppenna]

Description

Nanvix embraces an asymmetric kernel design. That is, when a user-level thread invokes a kernel call, that call is dispatched for execution as follows:

• If the kernel call does not have any side effects, it is executed by the underlying kernel thread that is associated to the user-level thread that made the call (local execution).
• If the kernel call does have side effects, it is dispatched for execution on a dedicated kernel thread that handles side effect kernel calls (remote execution).

We should change or implementation to meet this design. Currently, our implementation runs all kernel calls locally.

What should be done?

• Identify kernel calls that have side effects
• Move execution of side effect kernel calls to handle_syscall()
• Fix dispatch logic in do_kcall() accordingly

Related Files

https://github.com/nanvix/microkernel/blob/17de3a67c131a2085738cab7e28de2302918ceeb/src/kernel/kcall/mod.c

[ravixr]

I've thought of two potential solutions that I believe could address this issue. I'd appreciate some feedback on which approach might be more suitable. Additionally, I understand if neither of these solutions seems ideal and welcome any suggestions for improvement.

Solution 1

The first solution is straightforward, using a simple if statement within the syscall case in do_kcall(). Here's the proposed implementation:

    case NR_thread_create:
        if (process_get_curr() != KERNEL_PROCESS) {
            goto handle_side_effect;
        }
        ret = kcall_thread_create(
            (void (*)())arg0, (void *)arg1, (void (*)(void))arg2);
        break;
    ...
    handle_side_effect:
    default:
        // Copy kernel call parameters.
        scoreboard.kcall_nr = kcall_nr;
        scoreboard.arg0 = arg0;
        scoreboard.arg1 = arg1;
        scoreboard.arg2 = arg2;
        scoreboard.arg3 = arg3;
        scoreboard.arg4 = arg4;

        semaphore_up(kernel_semaphore);
        semaphore_down(user_semaphore);

        ret = scoreboard.ret;
        break;

Then, do_kcall() is called again within handle_syscall(), executed by a kernel thread:

noreturn void handle_syscall(void)
{
    while (true) {
        semaphore_down(kernel_semaphore);
        scoreboard.ret = do_kcall(scoreboard.arg0,
                                  scoreboard.arg1,
                                  scoreboard.arg2,
                                  scoreboard.arg3,
                                  scoreboard.arg4,
                                  scoreboard.kcall_nr);
        semaphore_up(user_semaphore);
    }
}

Solution 2

The second solution proposes a restructuring of syscall handling using a table of syscall functions and an argument list. Here's a detailed explanation:

In do_kcall(), the function would then accept an argument list and the kernel call number:

int do_kcall(word_t *args, word_t kcall_nr)
{
    int ret = -1;
    switch (kcall_nr) {
        case NR_spawn:
        ...
        case NR_thread_create:
            // Copy kernel call parameters.
            scoreboard.kcall_nr = kcall_nr;
            scoreboard.args = args;

            semaphore_up(kernel_semaphore);
            semaphore_down(user_semaphore);

            ret = scoreboard.ret;
            break;
        default:
            ret = kcalltab[kcall_nr](args);
            break;            
    }

    return (ret);
}

In handle_syscall(), a kernel call is made directly from kcalltab:

noreturn void handle_syscall(void)
{
    while (true) {
        semaphore_down(kernel_semaphore);
        scoreboard.ret = kcalltab[scoreborad.kcall_nr](scoreboard.args);
        semaphore_up(user_semaphore);
    }
}

However, this approach needs a refactoring of kcall_ prefixed functions to retrieve arguments, as exemplified here:

tid_t kcall_thread_create(word_t *args)
{
    return (thread_create(process_get_curr(),
                          (void (*)())args[0],
                          (void *)args[1],
                          (void (*)(void))args[2]));
}

Furthermore, user-space syscall invocations would also require refactoring in libnanvix.

[ppenna]

@ravixr It is simpler than both solutions that were presented: if the number of the system call matches one within the set of system calls with side effects, you just fall through the default case.

Check out legacy code for reference: https://github.com/nanvix/microkernel/blob/fa34cdc75a5ed2bef296f1b25b7f6126fafbb6ee/src/kernel/sys/syscalls.c