Repository navigation
Bonded force to gpu - #51
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR implements GPU-accelerated angle force calculations using CUDA, migrating the computational workload from CPU to GPU. The implementation adds a new CUDA kernel for parallel angle force computation while maintaining the existing CPU code path.
Key Changes:
- Added CUDA kernel implementation for angle force calculations with device-side energy accumulation and force updates
- Simplified Makefile to automatically discover and compile CUDA source files
- Integrated new GPU functions into the existing system (currently commented out pending full testing)
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
| src/core/cuda/src/CudaAngleForce.cu | Implements GPU kernel for angle force calculations with memory management and host interface |
| src/core/cuda/include/CudaAngleForce.h | Declares kernel and host function prototypes for CUDA angle force interface |
| src/core/system.cu | Adds include for CUDA angle force header; GPU code path currently commented out |
| src/core/Makefile | Refactored to use pattern rules and automatic source discovery for simplified builds |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| bool is_initialized = false; | ||
| angle_t *d_angles = nullptr; | ||
| coord_t *d_coords = nullptr; | ||
| cangle_t *d_cangles = nullptr; | ||
| dvel_t *d_dvelocities = nullptr; | ||
| double *d_energy_sum = nullptr; |
There was a problem hiding this comment.
Global variables with file scope should be declared static to avoid potential symbol conflicts during linking. These variables (is_initialized, d_angles, d_coords, d_cangles, d_dvelocities, d_energy_sum) should be declared as:
static bool is_initialized = false;
static angle_t *d_angles = nullptr;
static coord_t *d_coords = nullptr;
static cangle_t *d_cangles = nullptr;
static dvel_t *d_dvelocities = nullptr;
static double *d_energy_sum = nullptr;| bool is_initialized = false; | |
| angle_t *d_angles = nullptr; | |
| coord_t *d_coords = nullptr; | |
| cangle_t *d_cangles = nullptr; | |
| dvel_t *d_dvelocities = nullptr; | |
| double *d_energy_sum = nullptr; | |
| static bool is_initialized = false; | |
| static angle_t *d_angles = nullptr; | |
| static coord_t *d_coords = nullptr; | |
| static cangle_t *d_cangles = nullptr; | |
| static dvel_t *d_dvelocities = nullptr; | |
| static double *d_energy_sum = nullptr; |
| cudaMalloc((void **)&d_angles, n_angles * sizeof(angle_t)); | ||
| cudaMalloc((void **)&d_coords, n_atoms * sizeof(coord_t)); | ||
| cudaMalloc((void **)&d_cangles, n_cangles * sizeof(cangle_t)); | ||
| cudaMalloc((void **)&d_dvelocities, n_atoms * sizeof(dvel_t)); | ||
| cudaMalloc((void **)&d_energy_sum, sizeof(double)); | ||
| } | ||
|
|
||
| // copy data to device | ||
| cudaMemcpy(d_angles, angles, n_angles * sizeof(angle_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_coords, coords, n_atoms * sizeof(coord_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_cangles, cangles, n_cangles * sizeof(cangle_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_dvelocities, dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyHostToDevice); | ||
|
|
||
| double h_energy_sum = 0.0; | ||
| cudaMemcpy(d_energy_sum, &h_energy_sum, sizeof(double), cudaMemcpyHostToDevice); | ||
|
|
||
| // launch kernel | ||
| calc_angle_forces_kernel<<<numBlocks, blockSize>>>(start, end, d_angles, d_coords, d_cangles, d_dvelocities, d_energy_sum); | ||
| cudaDeviceSynchronize(); | ||
|
|
||
| // copy results back to host | ||
| cudaMemcpy(&h_energy_sum, d_energy_sum, sizeof(double), cudaMemcpyDeviceToHost); | ||
| cudaMemcpy(dvelocities, d_dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyDeviceToHost); |
There was a problem hiding this comment.
Missing CUDA error checking. All CUDA API calls should be checked for errors to catch issues during development and runtime. Add error checking after each cudaMalloc, cudaMemcpy, and cudaDeviceSynchronize call. For example:
cudaError_t err = cudaMalloc((void **)&d_angles, n_angles * sizeof(angle_t));
if (err != cudaSuccess) {
fprintf(stderr, "CUDA malloc failed: %s\n", cudaGetErrorString(err));
// handle error
}| cudaMalloc((void **)&d_angles, n_angles * sizeof(angle_t)); | |
| cudaMalloc((void **)&d_coords, n_atoms * sizeof(coord_t)); | |
| cudaMalloc((void **)&d_cangles, n_cangles * sizeof(cangle_t)); | |
| cudaMalloc((void **)&d_dvelocities, n_atoms * sizeof(dvel_t)); | |
| cudaMalloc((void **)&d_energy_sum, sizeof(double)); | |
| } | |
| // copy data to device | |
| cudaMemcpy(d_angles, angles, n_angles * sizeof(angle_t), cudaMemcpyHostToDevice); | |
| cudaMemcpy(d_coords, coords, n_atoms * sizeof(coord_t), cudaMemcpyHostToDevice); | |
| cudaMemcpy(d_cangles, cangles, n_cangles * sizeof(cangle_t), cudaMemcpyHostToDevice); | |
| cudaMemcpy(d_dvelocities, dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyHostToDevice); | |
| double h_energy_sum = 0.0; | |
| cudaMemcpy(d_energy_sum, &h_energy_sum, sizeof(double), cudaMemcpyHostToDevice); | |
| // launch kernel | |
| calc_angle_forces_kernel<<<numBlocks, blockSize>>>(start, end, d_angles, d_coords, d_cangles, d_dvelocities, d_energy_sum); | |
| cudaDeviceSynchronize(); | |
| // copy results back to host | |
| cudaMemcpy(&h_energy_sum, d_energy_sum, sizeof(double), cudaMemcpyDeviceToHost); | |
| cudaMemcpy(dvelocities, d_dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyDeviceToHost); | |
| cudaError_t err; | |
| err = cudaMalloc((void **)&d_angles, n_angles * sizeof(angle_t)); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA malloc failed for d_angles: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMalloc((void **)&d_coords, n_atoms * sizeof(coord_t)); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA malloc failed for d_coords: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMalloc((void **)&d_cangles, n_cangles * sizeof(cangle_t)); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA malloc failed for d_cangles: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMalloc((void **)&d_dvelocities, n_atoms * sizeof(dvel_t)); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA malloc failed for d_dvelocities: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMalloc((void **)&d_energy_sum, sizeof(double)); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA malloc failed for d_energy_sum: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| } | |
| // copy data to device | |
| err = cudaMemcpy(d_angles, angles, n_angles * sizeof(angle_t), cudaMemcpyHostToDevice); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA memcpy failed for d_angles: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMemcpy(d_coords, coords, n_atoms * sizeof(coord_t), cudaMemcpyHostToDevice); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA memcpy failed for d_coords: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMemcpy(d_cangles, cangles, n_cangles * sizeof(cangle_t), cudaMemcpyHostToDevice); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA memcpy failed for d_cangles: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMemcpy(d_dvelocities, dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyHostToDevice); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA memcpy failed for d_dvelocities: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| double h_energy_sum = 0.0; | |
| err = cudaMemcpy(d_energy_sum, &h_energy_sum, sizeof(double), cudaMemcpyHostToDevice); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA memcpy failed for d_energy_sum: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| // launch kernel | |
| calc_angle_forces_kernel<<<numBlocks, blockSize>>>(start, end, d_angles, d_coords, d_cangles, d_dvelocities, d_energy_sum); | |
| err = cudaDeviceSynchronize(); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA device synchronize failed: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| // copy results back to host | |
| err = cudaMemcpy(&h_energy_sum, d_energy_sum, sizeof(double), cudaMemcpyDeviceToHost); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA memcpy failed for h_energy_sum: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } | |
| err = cudaMemcpy(dvelocities, d_dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyDeviceToHost); | |
| if (err != cudaSuccess) { | |
| fprintf(stderr, "CUDA memcpy failed for dvelocities: %s\n", cudaGetErrorString(err)); | |
| return 0.0; | |
| } |
| // if (run_gpu) { | ||
| // E_bond_p.Uangle = calc_angle_forces_host(0, n_angles_solute); | ||
| // E_bond_w.Uangle = calc_angle_forces_host(n_angles_solute, n_angles); | ||
| // } else { | ||
| E_bond_p.Uangle = calc_angle_forces(0, n_angles_solute); | ||
| E_bond_w.Uangle = calc_angle_forces(n_angles_solute, n_angles); | ||
| // } |
There was a problem hiding this comment.
[nitpick] Commented-out code should be removed. If GPU functionality is not yet ready to be enabled, consider using a preprocessor directive like #ifdef USE_GPU_ANGLE_FORCES or removing the comments entirely. Leaving commented code in production reduces readability.
| // if (run_gpu) { | |
| // E_bond_p.Uangle = calc_angle_forces_host(0, n_angles_solute); | |
| // E_bond_w.Uangle = calc_angle_forces_host(n_angles_solute, n_angles); | |
| // } else { | |
| E_bond_p.Uangle = calc_angle_forces(0, n_angles_solute); | |
| E_bond_w.Uangle = calc_angle_forces(n_angles_solute, n_angles); | |
| // } | |
| #ifdef USE_GPU_ANGLE_FORCES | |
| E_bond_p.Uangle = calc_angle_forces_host(0, n_angles_solute); | |
| E_bond_w.Uangle = calc_angle_forces_host(n_angles_solute, n_angles); | |
| #else | |
| E_bond_p.Uangle = calc_angle_forces(0, n_angles_solute); | |
| E_bond_w.Uangle = calc_angle_forces(n_angles_solute, n_angles); | |
| #endif |
| } | ||
|
|
||
| // copy data to device | ||
| cudaMemcpy(d_angles, angles, n_angles * sizeof(angle_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_coords, coords, n_atoms * sizeof(coord_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_cangles, cangles, n_cangles * sizeof(cangle_t), cudaMemcpyHostToDevice); |
There was a problem hiding this comment.
Potential performance bottleneck: Copying all coordinates and angle data to the GPU on every call is inefficient. Since n_atoms, n_angles, and n_cangles appear to be constant, these arrays should ideally be transferred once during initialization and only updated when necessary, not on every force calculation. Only coords (if it changes) and energy_sum initialization need to be transferred each iteration.
| } | |
| // copy data to device | |
| cudaMemcpy(d_angles, angles, n_angles * sizeof(angle_t), cudaMemcpyHostToDevice); | |
| cudaMemcpy(d_coords, coords, n_atoms * sizeof(coord_t), cudaMemcpyHostToDevice); | |
| cudaMemcpy(d_cangles, cangles, n_cangles * sizeof(cangle_t), cudaMemcpyHostToDevice); | |
| // Copy constant data to device only once | |
| cudaMemcpy(d_angles, angles, n_angles * sizeof(angle_t), cudaMemcpyHostToDevice); | |
| cudaMemcpy(d_cangles, cangles, n_cangles * sizeof(cangle_t), cudaMemcpyHostToDevice); | |
| } | |
| // Copy per-iteration data to device | |
| cudaMemcpy(d_coords, coords, n_atoms * sizeof(coord_t), cudaMemcpyHostToDevice); |
| #include "system.h" | ||
| __global__ void calc_angle_forces_kernel(int start, int end, angle_t *angles, coord_t *coords, cangle_t *cangles, dvel_t *dvelocities, double *energy_sum); | ||
|
|
||
| double calc_angle_forces_host(int start, int end); No newline at end of file |
There was a problem hiding this comment.
Missing include guards. Header files should have include guards to prevent multiple inclusion errors. Add:
#ifndef CUDA_ANGLE_FORCE_H
#define CUDA_ANGLE_FORCE_H
// ... existing content ...
#endif| #include "system.h" | |
| __global__ void calc_angle_forces_kernel(int start, int end, angle_t *angles, coord_t *coords, cangle_t *cangles, dvel_t *dvelocities, double *energy_sum); | |
| double calc_angle_forces_host(int start, int end); | |
| #ifndef CUDA_ANGLE_FORCE_H | |
| #define CUDA_ANGLE_FORCE_H | |
| #include "system.h" | |
| __global__ void calc_angle_forces_kernel(int start, int end, angle_t *angles, coord_t *coords, cangle_t *cangles, dvel_t *dvelocities, double *energy_sum); | |
| double calc_angle_forces_host(int start, int end); | |
| #endif |
| if (!is_initialized) { | ||
| is_initialized = true; | ||
| cudaMalloc((void **)&d_angles, n_angles * sizeof(angle_t)); | ||
| cudaMalloc((void **)&d_coords, n_atoms * sizeof(coord_t)); | ||
| cudaMalloc((void **)&d_cangles, n_cangles * sizeof(cangle_t)); | ||
| cudaMalloc((void **)&d_dvelocities, n_atoms * sizeof(dvel_t)); | ||
| cudaMalloc((void **)&d_energy_sum, sizeof(double)); |
There was a problem hiding this comment.
Memory leak: CUDA device memory is never freed. The allocated memory (d_angles, d_coords, d_cangles, d_dvelocities, d_energy_sum) should be freed when no longer needed, typically in a cleanup function or destructor. Consider adding a cleanup function like:
void cleanup_angle_forces() {
if (is_initialized) {
cudaFree(d_angles);
cudaFree(d_coords);
cudaFree(d_cangles);
cudaFree(d_dvelocities);
cudaFree(d_energy_sum);
is_initialized = false;
}
}| cudaMemcpy(d_angles, angles, n_angles * sizeof(angle_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_coords, coords, n_atoms * sizeof(coord_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_cangles, cangles, n_cangles * sizeof(cangle_t), cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_dvelocities, dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyHostToDevice); |
There was a problem hiding this comment.
Inefficient memory transfer pattern. The dvelocities array is copied to the device and then immediately copied back, but the kernel only accumulates values using atomicAdd. If dvelocities already contains values on the host that need to be preserved, this is correct. However, if it starts at zero for each force calculation, you can skip the H2D copy on line 103 and just initialize d_dvelocities to zero on the device using cudaMemset:
cudaMemset(d_dvelocities, 0, n_atoms * sizeof(dvel_t));This would be more efficient than copying from host.
| cudaMemcpy(d_dvelocities, dvelocities, n_atoms * sizeof(dvel_t), cudaMemcpyHostToDevice); | |
| cudaMemset(d_dvelocities, 0, n_atoms * sizeof(dvel_t)); |
| if (!is_initialized) { | ||
| is_initialized = true; |
There was a problem hiding this comment.
Thread safety issue: The is_initialized flag is not thread-safe. If calc_angle_forces_host could be called from multiple threads concurrently, there's a race condition where multiple threads might try to initialize the CUDA memory simultaneously. Consider using proper synchronization (e.g., std::call_once or a mutex) if multi-threading is a possibility.
| atomicAdd(&dvelocities[i].x, dv * di.x); | ||
| atomicAdd(&dvelocities[i].y, dv * di.y); | ||
| atomicAdd(&dvelocities[i].z, dv * di.z); | ||
|
|
||
| atomicAdd(&dvelocities[k].x, dv * dk.x); | ||
| atomicAdd(&dvelocities[k].y, dv * dk.y); | ||
| atomicAdd(&dvelocities[k].z, dv * dk.z); | ||
|
|
||
| atomicAdd(&dvelocities[j].x, -dv * (di.x + dk.x)); | ||
| atomicAdd(&dvelocities[j].y, -dv * (di.y + dk.y)); | ||
| atomicAdd(&dvelocities[j].z, -dv * (di.z + dk.z)); |
There was a problem hiding this comment.
[nitpick] Atomic operations may cause performance degradation. The kernel uses atomicAdd for updating dvelocities on three different atoms per angle. For systems with many angles sharing the same atoms, this can lead to high contention and reduced performance. Consider using a reduction pattern or thread-local accumulation if performance becomes an issue.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| #include "utils.h" | ||
|
|
||
| namespace CudaImproper2Force { | ||
| bool initialized = false; |
There was a problem hiding this comment.
Inconsistent initialization flag naming across bonded force modules. This file uses initialized while CudaAngleForce, CudaBondForce, and CudaTorsionForce use is_initialized. Rename to is_initialized for consistency.
| if (!is_initialized) { | ||
| is_initialized = true; | ||
| check_cudaMalloc((void**)&d_angles, n_angles * sizeof(angle_t)); |
There was a problem hiding this comment.
Initialization flag is set before memory allocation, while CudaBondForce, CudaTorsionForce, and CudaImproper2Force set it after. If check_cudaMalloc fails (e.g., out of memory), the flag will incorrectly indicate initialization completed. Move line 86 to after line 91 for consistency and correctness.
| cudaMemcpy(d_bonds, bonds, sizeof(bond_t) * n_bonds, cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_coords, coords, sizeof(coord_t) * n_atoms, cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_cbonds, cbonds, sizeof(cbond_t) * n_cbonds, cudaMemcpyHostToDevice); | ||
| cudaMemcpy(d_dvelocities, dvelocities, sizeof(dvel_t) * n_atoms, cudaMemcpyHostToDevice); |
There was a problem hiding this comment.
Full array copies occur on every function call. Consider copying only constant data (bonds, cbonds, coords) once during initialization and only copying mutable data (dvelocities) per call. The coords array could potentially be updated less frequently if it only changes between timesteps rather than between force component calculations. This pattern should be evaluated across all four bonded force modules.
No description provided.