openmp中parallel for处理双重循环遇到问题的分析和解决方案

写在前面

同学的程序遇到问题，主要是使用parallel for的双重循环，代码结果一直计算不对，但是一直没找到原因。后来发现原因挺简单的，就是在双重循环的时候，假如第一层是遍历 i，第二层是遍历 j，这个 j 如果是在omp_set_num_threads()之前声明的，就说明 j 被多个线程共用，那么可能这个线程正在使用 j，然后被另一个线程更改了。

omp parallel for

测试程序：

#include <omp.h>
#include <stdio.h>
using namespace std;

#define NUM_THREADS 2

int main()
{
    int i;
    printf("initila i address is %p\n", &i);
    
    omp_set_num_threads(NUM_THREADS);

    #pragma omp parallel for
    for (i = 0; i < 10; ++i)
    {
        printf("thread is %d, i=%d, i address is %p\n", 
                omp_get_thread_num(), i, &i);
    }

    return 0;
}

在开辟线程之前声明了 i，执行程序，结果如下：

initila i address is 0x7ffc060c5034
thread is 0, i=0, i address is 0x7ffc060c4fc4
thread is 0, i=1, i address is 0x7ffc060c4fc4
thread is 0, i=2, i address is 0x7ffc060c4fc4
thread is 0, i=3, i address is 0x7ffc060c4fc4
thread is 0, i=4, i address is 0x7ffc060c4fc4
thread is 1, i=5, i address is 0x7ff8e59dde04
thread is 1, i=6, i address is 0x7ff8e59dde04
thread is 1, i=7, i address is 0x7ff8e59dde04
thread is 1, i=8, i address is 0x7ff8e59dde04
thread is 1, i=9, i address is 0x7ff8e59dde04

可以看到，虽然 i 在开辟线程之前声明，但是两个线程中的 i 都是新的i，地址不同。

如果我们把程序改成双重循环，程序如下所示：

#include <omp.h>
#include <stdio.h>
#include <iostream>
using namespace std;

#define NUM_THREADS 2

int main()
{
    int i, j;
    printf("initila i address is %p, initial j address is %p\n", &i, &j);
    
    omp_set_num_threads(NUM_THREADS);

    #pragma omp parallel for
    for (i = 0; i < 4; ++i)
    {
        for (j = 0; j < 1; ++j)
        {
            printf("thread is %d, i=%d, i address is %p, j=%d, j address is %p\n", 
                    omp_get_thread_num(), i, &i, j, &j);
        }
    }

    return 0;
}

结果如下：

initila i address is 0x7ffd9bc701a8, initial j address is 0x7ffd9bc701ac
thread is 0, i=0, i address is 0x7ffd9bc70134, j=0, j address is 0x7ffd9bc701ac
thread is 0, i=1, i address is 0x7ffd9bc70134, j=0, j address is 0x7ffd9bc701ac
thread is 1, i=2, i address is 0x7f947b8e2e04, j=0, j address is 0x7ffd9bc701ac
thread is 1, i=3, i address is 0x7f947b8e2e04, j=0, j address is 0x7ffd9bc701ac

可以看到，一共两个线程，i 和 j 都是在开辟线程之前定义，但是openmp parallel for会为每个线程新创建一个 i，但是并不会新建一个 j，j 还是最开始创建的那个。这就导致双重循环中，第二重循环的 j 产生数据冲突。

解决方法

使用parallel for，只会把最外层的循环分给不同的线程，同一个外层循环的内层的循环只能由一个线程执行，要想实现同一个外层循环的内层循环也被多个线程执行，可以使用collapse。
我们可以写循环的时候，内存循环写成：for (int j = 0; j < xxx; ++j)，就可以为每个线程创建不同的 j 了，避免数据冲突。或者我们可以把#pragma omp parallel for(private j)，强制让每个线程新创建 j。

修改1

    #pragma omp parallel for
    for (i = 0; i < 4; ++i)
    {
        for (int j = 0; j < 1; ++j)
        {
            printf("thread is %d, i=%d, i address is %p, j=%d, j address is %p\n", 
                    omp_get_thread_num(), i, &i, j, &j);
        }
    }

结果：

initila i address is 0x7ffeddabd0b0, initial j address is 0x7ffeddabd0b4
thread is 0, i=0, i address is 0x7ffeddabd040, j=0, j address is 0x7ffeddabd044
thread is 0, i=1, i address is 0x7ffeddabd040, j=0, j address is 0x7ffeddabd044
thread is 1, i=2, i address is 0x7f8b40564e00, j=0, j address is 0x7f8b40564e04
thread is 1, i=3, i address is 0x7f8b40564e00, j=0, j address is 0x7f8b40564e04

修改2

    #pragma omp parallel for private(j)
    for (i = 0; i < 4; ++i)
    {
        for (j = 0; j < 1; ++j)
        {
            printf("thread is %d, i=%d, i address is %p, j=%d, j address is %p\n", 
                    omp_get_thread_num(), i, &i, j, &j);
        }
    }

结果：

initila i address is 0x7ffe42cfda00, initial j address is 0x7ffe42cfda04
thread is 0, i=0, i address is 0x7ffe42cfd990, j=0, j address is 0x7ffe42cfd994
thread is 0, i=1, i address is 0x7ffe42cfd990, j=0, j address is 0x7ffe42cfd994
thread is 1, i=2, i address is 0x7fc7c6054e00, j=0, j address is 0x7fc7c6054e04
thread is 1, i=3, i address is 0x7fc7c6054e00, j=0, j address is 0x7fc7c6054e04