并行处理 – 从多个MPI输出组成VTK文件

对于盖子驱动腔(CFD)的Lattice Boltzmann模拟,我将我的立方域分解为(也是立方体)8个子域,这些子域由8个等级独立计算.每个MPI等级为每个时间步产生一个VTK文件,因为我使用ParaView,我希望将整个事物可视化为一个立方体.更具体地说明我想要实现的目标:

>我有一个长度为8的立方体(每个方向的元素数量)=> 8x8x8 = 512个元素.
>每个维度分配到2个等级,即每个等级处理4x4x4 = 64个元素.
>每个等级将其结果写入VTK StructuredGrid格式的名为lbm_out_< rank>.< timestep> .vts的文件中.
>我想生成一个收集* .vts文件的.pvts文件,并将包含子域的文件合并到ParaView可以视为整个域的单个文件中.

不幸的是我面临很多问题,因为我觉得ParaView和VTK的文档记录非常糟糕,而ParaView的错误消息完全没用.

我有以下* .pvts文件,其中包含一个ghost层和:

<?xml version="1.0"?>
<VTKFile type="PStructuredGrid" version="0.1" byte_order="LittleEndian">
    <PStructuredGrid WholeExtent="0 7 0 7 0 7 " GhostLevel="1">
        <PPoints>
            <PDataArray NumberOfComponents="3" type="Float32" />
        </PPoints>
        <Piece Extent="0 4 0 4 0 4" Source="lbm_out_0.0.vts"/>
        <Piece Extent="3 7 0 4 0 4" Source="lbm_out_1.0.vts"/>
        <Piece Extent="0 4 3 7 0 4" Source="lbm_out_2.0.vts"/>
        <Piece Extent="3 7 3 7 0 4" Source="lbm_out_3.0.vts"/>
        <Piece Extent="0 4 0 4 3 7" Source="lbm_out_4.0.vts"/>
        <Piece Extent="3 7 0 4 3 7" Source="lbm_out_5.0.vts"/>
        <Piece Extent="0 4 3 7 3 7" Source="lbm_out_6.0.vts"/>
        <Piece Extent="3 7 3 7 3 7" Source="lbm_out_7.0.vts"/>
    </PStructuredGrid>
</VTKFile>

拥有该文件,我认为应该正常工作(请注意,还没有参数,只是简单的几何信息),我的域范围完全搞砸了,尽管每个* .vts文件本身都可以正常工作.我附上了截图,以便更清楚:

可能是什么问题?是否可以将遗留VTK文件用于此任务?我可能做错了吗?我真的不知道如何完成这项任务,我在谷歌找到的资源非常有限.谢谢.

最佳答案 不幸的是
vtkXMLPStructuredGridWriter级(
VTK Classes not used in the Examples)没有例子.因此,我决定编写最简单的代码,为结构化网格生成* .vts和.pvts文件,与您正在寻找的情况非常相似.

以下代码使用MPI和VTK编写并行结构化网格文件.在此示例中,我们有两个创建自己的.vts文件的进程,而vtkXMLPStructuredGridWriter类编写.pvts文件:

// MPI Library
#include <mpi.h>

//VTK Library
#include <vtkXMLPStructuredGridWriter.h>
#include <vtkStructuredGrid.h>
#include <vtkSmartPointer.h>
#include <vtkFloatArray.h>
#include <vtkCellData.h>
#include <vtkMPIController.h>
#include <vtkProgrammableFilter.h>
#include <vtkInformation.h>

struct Args {
  vtkProgrammableFilter* pf;
  int local_extent[6];
};

// function to operate on the point attribute data
void execute (void* arg) {
  Args* args = reinterpret_cast<Args*>(arg);
  auto info = args->pf->GetOutputInformation(0);
  auto output_tmp = args->pf->GetOutput();
  auto input_tmp  = args->pf->GetInput();
  vtkStructuredGrid* output = dynamic_cast<vtkStructuredGrid*>(output_tmp);
  vtkStructuredGrid* input  = dynamic_cast<vtkStructuredGrid*>(input_tmp);
  output->ShallowCopy(input);
  output->SetExtent(args->local_extent);
}

int main (int argc, char *argv[]) {
  MPI_Init (&argc, &argv);
  int myrank;
  MPI_Comm_rank (MPI_COMM_WORLD, &myrank);

  int lx {10}, ly{10}, lz{10};        //local dimensions of the process's grid
  int dims[3] = {lx+1, ly+1, lz+1};
  int global_extent[6] = {0, 2*lx, 0, ly, 0, lz};
  int local_extent[6] = {myrank*lx, (myrank+1)*lx, 0, ly, 0, lz};

  // Create and Initialize vtkMPIController
  auto contr = vtkSmartPointer<vtkMPIController>::New();
  contr->Initialize(&argc, &argv, 1);
  int nranks = contr->GetNumberOfProcesses();
  int rank   = contr->GetLocalProcessId();

  // Create grid points, allocate memory and Insert them
  auto points = vtkSmartPointer<vtkPoints>::New();
  points->Allocate(dims[0]*dims[1]*dims[2]);
  for (int k=0; k<dims[2]; ++k)
    for (int j=0; j<dims[1]; ++j)
      for (int i=0; i<dims[0]; ++i)
        points->InsertPoint(i + j*dims[0] + k*dims[0]*dims[1],
                            i+rank*(dims[0]-1), j, k);

  // Create a density field. Note that the number of cells is always less than
  // number of grid points by an amount of one so we use dims[i]-1
  auto density = vtkSmartPointer<vtkFloatArray>::New();
  density->SetNumberOfComponents(1);
  density->SetNumberOfTuples((dims[0]-1)*(dims[1]-1)*(dims[2]-1));
  density->SetName ("density");
  int index;
  for (int k=0; k<lz; ++k)
    for (int j=0; j<ly; ++j)
      for (int i=0; i<lx; ++i) {
        index = i + j*(dims[0]-1) + k*(dims[0]-1)*(dims[1]-1);
        density->SetValue(index, i+j+k);
      }

  // Create a vtkProgrammableFilter
  auto pf = vtkSmartPointer<vtkProgrammableFilter>::New();

  // Initialize an instance of Args
  Args args;
  args.pf = pf;
  for(int i=0; i<6; ++i) args.local_extent[i] = local_extent[i];

  pf->SetExecuteMethod(execute, &args);

  // Create a structured grid and assign point data and cell data to it
  auto structuredGrid = vtkSmartPointer<vtkStructuredGrid>::New();
  structuredGrid->SetExtent(global_extent);
  pf->SetInputData(structuredGrid);
  structuredGrid->SetPoints(points);
  structuredGrid->GetCellData()->AddArray(density);

  // Create the parallel writer and call some functions
  auto parallel_writer = vtkSmartPointer<vtkXMLPStructuredGridWriter>::New();
  parallel_writer->SetInputConnection(pf->GetOutputPort());
  parallel_writer->SetController(contr);
  parallel_writer->SetFileName("test.pvts");
  parallel_writer->SetNumberOfPieces(nranks);
  parallel_writer->SetStartPiece(rank);
  parallel_writer->SetEndPiece(rank);
  parallel_writer->SetDataModeToBinary();
  parallel_writer->Update();
  parallel_writer->Write();

  contr->Finalize();

  // WARNING: it seems that MPI_Finalize is not necessary since we are using
  // Finalize() function from vtkMPIController class. Uncomment the following
  // line and see what happens.
//   MPI_Finalize ();
  return 0;
}

此代码仅将一些数据(在本例中为密度,即标量)写入文件,而不进行渲染.对于可视化,您需要一个像Paraview这样的软件.
要运行此代码,您可以使用此CMake文件:

cmake_minimum_required(VERSION 2.8)

PROJECT(PXMLStructuredGridWriter)
add_executable(PXMLStructuredGridWriter parallel_vtk.cpp)

find_package(VTK REQUIRED)
include(${VTK_USE_FILE})
target_link_libraries(PXMLStructuredGridWriter ${VTK_LIBRARIES})

find_package(MPI REQUIRED)
include_directories(${MPI_INCLUDE_PATH})
target_link_libraries(PXMLStructuredGridWriter ${MPI_LIBRARIES})

最后,您将在具有可执行文件的同一目录中找到这样的xml文件:

<?xml version="1.0"?>
<VTKFile type="PStructuredGrid" version="0.1" byte_order="LittleEndian" header_type="UInt32" compressor="vtkZLibDataCompressor">
  <PStructuredGrid WholeExtent="0 20 0 10 0 10" GhostLevel="2">
    <PCellData>
      <PDataArray type="Float32" Name="density"/>
    </PCellData>
    <PPoints>
      <PDataArray type="Float32" Name="Points" NumberOfComponents="3"/>
    </PPoints>
    <Piece Extent="0 10 0 10 0 10" Source="test_0.vts"/>
    <Piece Extent="10 20 0 10 0 10" Source="test_1.vts"/>
  </PStructuredGrid>
</VTKFile>
点赞