Skip to content

Instantly share code, notes, and snippets.

@nicksantamaria
Created October 20, 2016 22:35
Show Gist options
  • Select an option

  • Save nicksantamaria/21dce5ff2a6640cdff76ce7bc57d2981 to your computer and use it in GitHub Desktop.

Select an option

Save nicksantamaria/21dce5ff2a6640cdff76ce7bc57d2981 to your computer and use it in GitHub Desktop.
Example: Parallel processing in PHP using pcntl_fork()
<?php
/**
* @file
* Basic demonstration of how to do parallel threads in PHP.
*/
// This array of "tasks" could be anything. For demonstration purposes
// these are just strings, but they could be a callback, class or
// include file (hell, even code-as-a-string to pass to eval()).
$tasks = [
"fetch_remote_data",
"post_async_updates",
"clear_caches",
"notify_admin",
];
// This loop creates a new fork for each of the items in $tasks.
foreach ($tasks as $task) {
$pid = pcntl_fork();
if ($pid == -1) {
exit("Error forking...\n");
}
else if ($pid == 0) {
execute_task($task);
exit();
}
}
// This while loop holds the parent process until all the child threads
// are complete - at which point the script continues to execute.
while(pcntl_waitpid(0, $status) != -1);
// You could have more code here.
echo "Do stuff after all parallel execution is complete.\n";
/**
* Helper method to execute a task.
*/
function execute_task($task_id) {
echo "Starting task: ${task_id}\n";
// Simulate doing actual work with sleep().
$execution_time = rand(5, 10);
sleep($execution_time);
echo "Completed task: ${task_id}. Took ${execution_time} seconds.\n";
}
@v0ff4k
Copy link
Copy Markdown

v0ff4k commented Mar 6, 2021

not work for me in the method of class. cause it try to execute deconstructors in every loop, so i got alot errors abous sql had gone away
and
Uncaught PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

thats all folks !
p.s. use Amphp, its better way !

@alexalouit
Copy link
Copy Markdown

not work for me in the method of class. cause it try to execute deconstructors in every loop, so i got alot errors abous sql had gone away
and
Uncaught PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

thats all folks !
p.s. use Amphp, its better way !

you must connect to database again in child process, this one lost connection during the fork.

@chung362
Copy link
Copy Markdown

Great tutorial , thank you

@williamdes
Copy link
Copy Markdown

Here is my improved version
Also posted as a note on https://www.php.net/manual/en/function.pcntl-fork.php

<?php

declare(strict_types = 1);

/**
 * Helper method to execute a task
 */
function execute_task(int $task_id): void
{
    echo 'Starting task: ' . $task_id . PHP_EOL;

    // Simulate doing actual work with sleep()
    $execution_time = rand(5, 10);
    sleep($execution_time);

    echo "Completed task: ${task_id}. Took ${execution_time} seconds.\n";
}

/**
 * Builds a list of tasks
 */
function generator(): Generator
{
    $item_count = 50;
    for ($i = 1; $i <= $item_count; $i++) {
        yield $i;
    }
}

/**
 * Starts the work
 */
function launch(): void
{
    $processCount = 0;
    $status       = null;
    echo 'Running as pid: ' . getmypid() . PHP_EOL;

    $taskList = generator();
    do {
        echo 'Still tasks to do' . PHP_EOL;

        if ($processCount >= 5) {
            echo 'Waiting, currently running: ' . $processCount . PHP_EOL;
            pcntl_wait($status);
            $processCount--;
            continue;
        }

        echo 'Tasks running: ' . $processCount . PHP_EOL;
        $task = $taskList->current();
        $taskList->next();
        $processCount++;

        $pid = pcntl_fork();

        if ($pid === -1) {
            exit('Error forking...' . PHP_EOL);
        }

        if ($pid === 0) {
            $processList[] = getmypid();
            echo 'Running task as pid: ' . getmypid() . PHP_EOL;
            execute_task($task);
            exit();
        }
    } while ($taskList->valid());

    $waitForChildrenToFinish = true;
    while ($waitForChildrenToFinish) {
        // This while loop holds the parent process until all the child threads
        // are complete - at which point the script continues to execute.
        if (pcntl_waitpid(0, $status) === -1) {
            echo 'parallel execution is complete' . PHP_EOL;
            $waitForChildrenToFinish = false;
        }
    }

    // Code to run after all tasks are processed
}

launch();

@shaddow11
Copy link
Copy Markdown

Great example! Pretty much the clearest one i've seen!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment