Lab 01
Vector operations on CPU (C)
Element-wise add, subtract and multiply, scalar with k = 12, dot product, and sum, max, min and average for N = 60.
Free · Read online · Parallel Computing · CSC334 · By Uzair Malik
A free study guide for Parallel and Distributed Computing (CSC334) at COMSATS Abbottabad, prepared by Uzair Malik. Every lab comes with complete and clean code with Roman Urdu comments, a line-by-line explanation in plain words, and a quick cheat-sheet showing exactly which line to change.
Download the PDF — free for subscribers
Reading online above is free for everyone. To download the PDF, subscribe below — one email a week, unsubscribe anytime.
The full guide, as readable text right here on this page — no PDF, no sign-up needed to read. The PDF download unlocks after subscribing.
Course: Parallel and Distributed Computing (CSC334) — BCS-8A/8B — Instructor: Rabbiya Hameed — COMSATS Abbottabad
Ye guide tumhari apni 7 pre-performed lab files (Lab 01 ta Lab 06 + Lab Assignment) se bani hai. Har task ke liye 4 cheezein di gayi hain: (1) Complete & Clean Code — Roman Urdu comments ke sath, (2) Bachon ki tarah asaan line-by-line report, (3) Mam exam mein kya change kar sakti hain — expected twists, (4) Quick Cheat-Sheet — kahan kya badalna hai, exact jagah ke sath.
EXAM KA GOLDEN RULE: Exam mein sawal inhi labs se SLIGHTLY CHANGE ho kar aayega. Matlab code ka DHAANCHA (structure) wohi rahega, sirf ek-do cheezein badlegi: number (N, k), operation (+ ki jagah -, sum ki jagah max), range (10-99 ki jagah kuch aur), ya naam. Is liye har task mein 'Cheat-Sheet' wala hissa sab se zyada important hai — wohi parho exam se ek raat pehle.
| Lab | Topic | Tasks |
|---|---|---|
| Lab 01 | Vector Operations on CPU (C language) | Q1 Element-wise Add/Sub/Mul, Q2 Scalar k=12, Q3 Dot Product, Q4 Sum/Max/Min/Avg |
| Lab 02 | Vector Operations on CPU + GPU (CUDA) | Wohi 4 tasks, magar GPU kernels mein — atomicAdd, shared memory |
| Lab 03 | Mutual Exclusion & Synchronization (C++ Threads) | Mutex Race Condition, Ping-Pong (Condition Variable), Semaphore |
| Lab 04 | Advanced Vector Operations on CPU | Smoothing, Thresholding, Histogram |
| Lab 05 | Advanced Vector Operations on GPU (CUDA) | Wohi 3 tasks GPU par — atomic histogram |
| Lab 06 | 2D Convolution in CUDA C | 512x512 image, 5x5 mask, constant memory + Home Task: Matrix x Vector |
| Assignment | Bara N (2^20, 2^24) — Timing + CUDA Histogram | CPU timing, Global index, Histogram ki 3 versions (race / atomic / shared) |
Ye 5 templates rat lo. Har lab ka 70% code inhi se banta hai. Exam mein change bhi aksar inhi lines mein aata hai.
srand(time(NULL)); // har dafa naye random numbers ke liye
for (int i = 0; i < N; i++) {
hA[i] = rand() % 90 + 10; // % 90 se 0..89 banta hai, +10 karne se 10..99
hB[i] = rand() % 90 + 10; // wohi kaam dusre vector ke liye
}// STEP 1: GPU par jagah lo (cudaMalloc)
// STEP 2: Data CPU se GPU bhejo (cudaMemcpy HostToDevice)
// STEP 3: Kernel chalao kernel<<<blocks, threads>>>(...)
// STEP 4: Result GPU se wapas CPU lao (cudaMemcpy DeviceToHost)
// STEP 5: GPU ki jagah khaali karo (cudaFree)int i = threadIdx.x; // CHOTA kaam: N=60, sirf 1 block — Lab 02/05 ke solved codes
int i = blockIdx.x * blockDim.x + threadIdx.x; // BARA kaam: bohat se blocks — Assignment / asal sahi tareeqa
if (i < N) { /* kaam sirf tab karo jab i hadd ke andar ho */ } // GUARD: fazool threads ko rokoMaqsad: Do vectors hA aur hB hain, har ek mein N=60 numbers hain, har number 10 se 99 ke darmiyan hai. In par hisaab kitab karna hai — pehle sirf CPU par, aam C code se, ek ke baad ek (sequential).
#include <stdio.h> // printf ke liye
#include <stdlib.h> // rand(), srand() ke liye
#include <time.h> // time() ke liye
#define N 60 // vector ka size = 60 numbers
#define K 12 // scalar constant k = 12
int main() {
int hA[N], hB[N], hC[N]; // hA, hB = input, hC = result rakhne ke liye
int hProd = 0; // dot product ka jawab (shuru mein 0)
int hSum = 0; // sum ka jawab (shuru mein 0)
srand(time(NULL)); // random numbers har dafa alag aayein
for (int i = 0; i < N; i++) { // 0 se 59 tak loop
hA[i] = rand() % 90 + 10; // 10..99 ka number hA mein
hB[i] = rand() % 90 + 10; // 10..99 ka number hB mein
}
// ---------- Q1: Element-wise Add / Sub / Mul ----------
for (int i = 0; i < N; i++) hC[i] = hA[i] + hB[i]; // jorna: hC = hA + hB
// Sub ke liye sirf + ki jagah - likhna hai, Mul ke liye * likhna hai
// ---------- Q2: Scalar (k=12 ke sath) ----------
for (int i = 0; i < N; i++) hC[i] = hA[i] * K; // har number * 12
// Add ke liye hA[i] + K, Sub ke liye hA[i] - K
// ---------- Q3: Dot Product ----------
for (int i = 0; i < N; i++) hProd += hA[i] * hB[i]; // jora guna karo, jama karte jao
// ---------- Q4: Sum / Max / Min / Avg ----------
int hMax = hA[0]; // pehle number ko hi max farz kar lo
int hMin = hA[0]; // pehle number ko hi min farz kar lo
for (int i = 0; i < N; i++) {
hSum += hA[i]; // har number jama karo
if (hA[i] > hMax) hMax = hA[i]; // bara number mile to max badal do
if (hA[i] < hMin) hMin = hA[i]; // chota mile to min badal do
}
double hAvg = (double)hSum / N; // average = sum / kul tadaad
printf("Dot Product = %d\n", hProd); // Q3 ka jawab print
printf("Sum=%d Max=%d Min=%d Avg=%.2f\n", hSum, hMax, hMin, hAvg); // Q4 print
return 0;
}Maqsad: Lab 01 wale wohi 4 kaam, magar is dafa GPU par. Farq sirf itna hai: CPU mein loop ek ek karke chalta tha, GPU mein har element ke liye EK THREAD ban jata hai — 60 threads ek sath kaam karte hain. Isi liye kernel mein loop NAHI hota, thread apna number (i) khud nikalta hai.
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#define N 60
#define K 12
// ----- Q1 KERNEL: element-wise add/sub/mul — har thread ek element -----
__global__ void addSubMul(int *dA, int *dB, int *dAdd, int *dSub, int *dMul) {
int i = blockIdx.x * blockDim.x + threadIdx.x; // mera global number
if (i < N) { // guard: fazool thread bahar rahe
dAdd[i] = dA[i] + dB[i]; // jorna
dSub[i] = dA[i] - dB[i]; // ghatana
dMul[i] = dA[i] * dB[i]; // guna karna
}
}
// ----- Q2 KERNEL: scalar k ke sath -----
__global__ void scalarOps(int *dA, int *dAdd, int *dSub, int *dMul) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < N) {
dAdd[i] = dA[i] + K; // har element + 12
dSub[i] = dA[i] - K; // har element - 12
dMul[i] = dA[i] * K; // har element * 12
}
}
// ----- Q3 KERNEL: dot product — atomicAdd ke sath -----
__global__ void dotProductAtomic(int *dA, int *dB, int *dProd) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < N) {
int temp = dA[i] * dB[i]; // pehle apna guna khud karo
atomicAdd(dProd, temp); // phir LINE mein lag kar gulak mein daalo
}
}
// ----- Q4 KERNEL: sum/max/min — atomic family ke sath -----
__global__ void statsAtomic(int *dA, int *dSum, int *dMax, int *dMin) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < N) {
atomicAdd(dSum, dA[i]); // sum: sab ek gulak mein, atomically
atomicMax(dMax, dA[i]); // max: bara ho to hi badlega
atomicMin(dMin, dA[i]); // min: chota ho to hi badlega
}
}
int main() {
int hA[N], hB[N], hAdd[N], hSub[N], hMul[N];
int hProd = 0, hSum = 0, hMax = 0, hMin = 999; // atomic ke liye shuruati values
int *dA, *dB, *dAdd, *dSub, *dMul, *dProd, *dSum, *dMax, *dMin;
int bytes = N * sizeof(int); // ek vector ki bytes mein size = 60*4 = 240
srand(time(NULL));
for (int i = 0; i < N; i++) { hA[i] = rand() % 90 + 10; hB[i] = rand() % 90 + 10; }
// STEP 1: GPU par jagah lo
cudaMalloc(&dA, bytes); cudaMalloc(&dB, bytes);
cudaMalloc(&dAdd, bytes); cudaMalloc(&dSub, bytes); cudaMalloc(&dMul, bytes);
cudaMalloc(&dProd, sizeof(int)); cudaMalloc(&dSum, sizeof(int));
cudaMalloc(&dMax, sizeof(int)); cudaMalloc(&dMin, sizeof(int));
// STEP 2: data bhejo (aur choti values ko unki shuruati value do)
cudaMemcpy(dA, hA, bytes, cudaMemcpyHostToDevice);
cudaMemcpy(dB, hB, bytes, cudaMemcpyHostToDevice);
cudaMemcpy(dProd, &hProd, sizeof(int), cudaMemcpyHostToDevice);
cudaMemcpy(dSum, &hSum, sizeof(int), cudaMemcpyHostToDevice);
cudaMemcpy(dMax, &hMax, sizeof(int), cudaMemcpyHostToDevice);
cudaMemcpy(dMin, &hMin, sizeof(int), cudaMemcpyHostToDevice);
// STEP 3: kernels chalao — block size 8, blocks = (60+7)/8 = 8
int threads = 8, blocks = (N + threads - 1) / threads;
addSubMul<<<blocks, threads>>>(dA, dB, dAdd, dSub, dMul);
dotProductAtomic<<<blocks, threads>>>(dA, dB, dProd);
statsAtomic<<<blocks, threads>>>(dA, dSum, dMax, dMin);
// STEP 4: results wapas lao
cudaMemcpy(hAdd, dAdd, bytes, cudaMemcpyDeviceToHost);
cudaMemcpy(&hProd, dProd, sizeof(int), cudaMemcpyDeviceToHost);
cudaMemcpy(&hSum, dSum, sizeof(int), cudaMemcpyDeviceToHost);
cudaMemcpy(&hMax, dMax, sizeof(int), cudaMemcpyDeviceToHost);
cudaMemcpy(&hMin, dMin, sizeof(int), cudaMemcpyDeviceToHost);
double hAvg = (double)hSum / N; // average CPU par hi nikalta hai
printf("Dot=%d Sum=%d Max=%d Min=%d Avg=%.2f\n", hProd, hSum, hMax, hMin, hAvg);
// STEP 5: GPU jagah khaali karo
cudaFree(dA); cudaFree(dB); cudaFree(dAdd); cudaFree(dSub); cudaFree(dMul);
cudaFree(dProd); cudaFree(dSum); cudaFree(dMax); cudaFree(dMin);
return 0;
}Ye lab GPU wali nahi hai — ye aam CPU par kai THREADS chala kar un ko apas mein ladne se bachane ki lab hai. Teen hathiyaar hain: (1) Mutex — ek waqt mein ek thread, (2) Condition Variable — bari bari (turn by turn), (3) Semaphore — ek waqt mein ginay hue threads (jaise 2).
#include <iostream>
#include <thread> // thread banane ke liye
#include <mutex> // mutex (taala) ke liye
using namespace std;
int shared_var = 0; // DONO threads ki mushtarika cheez — yahin jhagra hota hai
mutex mtx; // TAALA: jis ke paas hoga, wohi andar jayega
void increment() {
for (int i = 0; i < 10000; i++) { // har thread 10000 dafa +1 karega
mtx.lock(); // taala lagao — dusra thread yahin ruk jayega
shared_var++; // CRITICAL SECTION: andar sirf EK thread hai
mtx.unlock(); // taala kholo — ab dusra aa sakta hai
}
}
int main() {
thread t1(increment); // pehla thread shuru
thread t2(increment); // dusra thread shuru
t1.join(); // t1 ke khatam hone ka intezaar
t2.join(); // t2 ke khatam hone ka intezaar
cout << "Final value: " << shared_var << endl; // hamesha 20000 aayega
return 0;
}#include <iostream>
#include <thread>
#include <mutex>
#include <condition_variable> // cv ke liye
using namespace std;
mutex mtx;
condition_variable cv;
bool ping_turn = true; // pehle Ping ki bari (true = Ping)
void printPing() {
for (int i = 0; i < 5; i++) {
unique_lock<mutex> lock(mtx); // taala pakro
cv.wait(lock, [] { return ping_turn; }); // apni bari tak SOYE raho
cout << "Ping "; // bari mili — print karo
ping_turn = false; // ab Pong ki bari
cv.notify_all(); // sab ko jaga do
}
}
void printPong() {
for (int i = 0; i < 5; i++) {
unique_lock<mutex> lock(mtx);
cv.wait(lock, [] { return !ping_turn; }); // Pong ki bari tab jab ping_turn false ho
cout << "Pong" << endl;
ping_turn = true; // wapas Ping ki bari
cv.notify_all();
}
}
int main() {
thread t1(printPing);
thread t2(printPong);
t1.join(); t2.join();
return 0;
} // Output: Ping Pong Ping Pong ... (5 dafa)#include <iostream>
#include <thread>
#include <chrono> // sleep ke liye
#include <semaphore.h> // POSIX semaphore: sem_t, sem_wait, sem_post
using namespace std;
sem_t semaphore; // TOKENS wali tokri — kitne andar ja sakte hain
void accessResource(int thread_id) {
sem_wait(&semaphore); // token lo — tokri khaali ho to yahin RUK jao
cout << "Thread " << thread_id << " is accessing the resource." << endl;
this_thread::sleep_for(chrono::seconds(2)); // 2 second kaam ka natak
cout << "Thread " << thread_id << " is done." << endl;
sem_post(&semaphore); // token wapas tokri mein — agla andar aa sakta hai
}
int main() {
sem_init(&semaphore, 0, 2); // shuru mein 2 tokens = ek waqt mein 2 threads
thread t1(accessResource, 1);
thread t2(accessResource, 2);
thread t3(accessResource, 3);
thread t4(accessResource, 4);
t1.join(); t2.join(); t3.join(); t4.join();
sem_destroy(&semaphore); // aakhir mein semaphore khatam
return 0;
}Teen naye kaam hain: Smoothing (padosi numbers ka average), Thresholding (number ko us ke group ka number 0/1/2/3 dena), Histogram (har group mein kitne numbers aaye, gintee karna). Teeno ka khaana peena ek hi hai: N=60, numbers 10-99, groups ye hain: [10-31]=bin0, [32-54]=bin1, [55-76]=bin2, [77-99]=bin3.
GROUPS YAAD KARNE KA TRICK: Pehla group 10 se shuru. Har group ki upar wali hadd: 31, 54, 76, 99. Code mein hum < 32, < 55, < 77 likhte hain — matlab '32 se chota hai to bin 0'. Dono ek hi baat hai.
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#define N 60
int main() {
int hA[N], hC[N]; // hA = asal numbers, hC = jawab
int hB1 = 0, hB2 = 0, hB3 = 0, hB4 = 0; // histogram ke 4 dabbe (bins)
srand(time(NULL));
for (int i = 0; i < N; i++) hA[i] = rand() % 90 + 10; // 10..99
// ---------- Q1: SMOOTHING ----------
hC[0] = hA[0]; // pehla element WESA KA WESA (us ka left padosi nahi)
hC[N-1] = hA[N-1]; // aakhri element bhi wesa ka wesa (right padosi nahi)
for (int i = 1; i < N-1; i++) // beech wale: 1 se N-2 tak
hC[i] = (hA[i-1] + hA[i] + hA[i+1]) / 3; // left + khud + right, divide 3
// ---------- Q2: THRESHOLDING ----------
for (int i = 0; i < N; i++) {
if (hA[i] < 32) hC[i] = 0; // 10..31 -> 0
else if (hA[i] < 55) hC[i] = 1; // 32..54 -> 1
else if (hA[i] < 77) hC[i] = 2; // 55..76 -> 2
else hC[i] = 3; // 77..99 -> 3
}
// ---------- Q3: HISTOGRAM ----------
for (int i = 0; i < N; i++) {
if (hA[i] < 32) hB1++; // bin1 mein ek aur
else if (hA[i] < 55) hB2++; // bin2 mein ek aur
else if (hA[i] < 77) hB3++; // bin3 mein ek aur
else hB4++; // bin4 mein ek aur
}
printf("Bin1=%d Bin2=%d Bin3=%d Bin4=%d (kul = %d)\n",
hB1, hB2, hB3, hB4, hB1+hB2+hB3+hB4); // kul hamesha N=60 hona chahiye
return 0;
}Lab 04 wale wohi 3 kaam, ab GPU par. Soch bilkul Lab 02 wali hai: loop khatam, har element ka ek thread. Do khaas baatein: histogram mein saare threads ek hi 4 dabbo ko badalte hain, is liye atomicAdd ke baghair race condition hoga; aur smoothing mein kernel ke andar boundary wala hisaab thread khud karta hai.
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#define N 60
// Q1: Smoothing kernel — har thread apna element smooth karta hai
__global__ void smoothVector(int *dA, int *dC) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= N) return; // guard
if (i == 0 || i == N-1) dC[i] = dA[i]; // kinare wale as-is copy
else dC[i] = (dA[i-1] + dA[i] + dA[i+1]) / 3; // beech wale: 3 ka average
}
// Q2: Thresholding kernel — har thread apne number ka bin number likhta hai
__global__ void thresholdVector(int *dA, int *dC) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= N) return;
int val = dA[i]; // pehle apna number uthao
if (val < 32) dC[i] = 0; // 10..31
else if (val < 55) dC[i] = 1; // 32..54
else if (val < 77) dC[i] = 2; // 55..76
else dC[i] = 3; // 77..99
}
// Q3: Histogram kernel — atomicAdd se race khatam
__global__ void histogramVector(int *dA, int *dB1, int *dB2, int *dB3, int *dB4) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= N) return;
int val = dA[i];
if (val < 32) atomicAdd(dB1, 1); // dabba 1 mein +1, line bana kar
else if (val < 55) atomicAdd(dB2, 1); // dabba 2 mein +1
else if (val < 77) atomicAdd(dB3, 1); // dabba 3 mein +1
else atomicAdd(dB4, 1); // dabba 4 mein +1
}
int main() {
int hA[N], hC[N];
int hB1 = 0, hB2 = 0, hB3 = 0, hB4 = 0, *dA, *dC, *dB1, *dB2, *dB3, *dB4;
int bytes = N * sizeof(int);
srand(time(NULL));
for (int i = 0; i < N; i++) hA[i] = rand() % 90 + 10;
cudaMalloc(&dA, bytes); cudaMalloc(&dC, bytes); // arrays ki jagah
cudaMalloc(&dB1, sizeof(int)); cudaMalloc(&dB2, sizeof(int)); // har bin ki jagah
cudaMalloc(&dB3, sizeof(int)); cudaMalloc(&dB4, sizeof(int));
cudaMemcpy(dA, hA, bytes, cudaMemcpyHostToDevice); // data bhejo
cudaMemcpy(dB1, &hB1, sizeof(int), cudaMemcpyHostToDevice); // bins ko 0 se shuru karo
cudaMemcpy(dB2, &hB2, sizeof(int), cudaMemcpyHostToDevice);
cudaMemcpy(dB3, &hB3, sizeof(int), cudaMemcpyHostToDevice);
cudaMemcpy(dB4, &hB4, sizeof(int), cudaMemcpyHostToDevice);
int threads = 8, blocks = (N + threads - 1) / threads; // 8 blocks x 8 = 64 threads
smoothVector<<<blocks, threads>>>(dA, dC);
cudaMemcpy(hC, dC, bytes, cudaMemcpyDeviceToHost); // smooth result wapas
histogramVector<<<blocks, threads>>>(dA, dB1, dB2, dB3, dB4);
cudaMemcpy(&hB1, dB1, sizeof(int), cudaMemcpyDeviceToHost); // ginteeyan wapas
cudaMemcpy(&hB2, dB2, sizeof(int), cudaMemcpyDeviceToHost);
cudaMemcpy(&hB3, dB3, sizeof(int), cudaMemcpyDeviceToHost);
cudaMemcpy(&hB4, dB4, sizeof(int), cudaMemcpyDeviceToHost);
printf("Bins: %d %d %d %d (kul=%d)\n", hB1, hB2, hB3, hB4, hB1+hB2+hB3+hB4);
cudaFree(dA); cudaFree(dC); cudaFree(dB1); cudaFree(dB2); cudaFree(dB3); cudaFree(dB4);
return 0;
}Ab tak hum 1D vector par the, ab 2D IMAGE hai: 512x512 grayscale tasveer. Convolution matlab har pixel ko us ke aas paas ke pixels ke sath ek MASK (choti 5x5 jadvali / filter) se guna-jama karke nayi tasveer banana — jaise photo par blur ya sharpen wala filter lagana. Mask sab threads ke liye ek hi hai aur sirf parhna hai, is liye usay CONSTANT MEMORY mein rakhte hain (tez bhi, sab ko ek sath broadcast bhi ho jata hai).
#include <stdio.h>
#include <stdlib.h>
#define MASK_WIDTH 5 // mask 5x5 ka hai
#define WIDTH 512
#define HEIGHT 512
__constant__ float d_M[MASK_WIDTH * MASK_WIDTH]; // mask CONSTANT memory mein (sirf GPU wala naam)
// Har thread EK output pixel banata hai
__global__ void Convolution2D_Kernel(float *d_N, float *d_P, int ImgWidth, int ImgHeight) {
int row = blockIdx.y * blockDim.y + threadIdx.y; // mera row (y wali line)
int col = blockIdx.x * blockDim.x + threadIdx.x; // mera column (x wali line)
if (row >= ImgHeight || col >= ImgWidth) return; // guard: tasveer se bahar wale threads ruk jao
float sum = 0.0f; // is pixel ka jawab jama hoga yahan
int radius = MASK_WIDTH / 2; // 5/2 = 2 (mask ka centre)
for (int m = 0; m < MASK_WIDTH; m++) { // mask ki rows par loop
for (int n = 0; n < MASK_WIDTH; n++) { // mask ki columns par loop
int r = row + m - radius; // tasveer mein woh row jo mask ke neeche hai
int c = col + n - radius; // tasveer mein woh column
float image_val = 0.0f; // zero-padding: bahar ho to 0 samjho
if (r >= 0 && r < ImgHeight && c >= 0 && c < ImgWidth)
image_val = d_N[r * ImgWidth + c]; // 2D -> 1D: row*width + col
sum += image_val * d_M[m * MASK_WIDTH + n]; // guna karke jama
}
}
d_P[row * ImgWidth + col] = sum; // jawab output tasveer mein likh do
}
int main() {
int size = WIDTH * HEIGHT; // 512*512 = 262144 pixels
float *h_N = (float*)malloc(size * sizeof(float)); // input tasveer (CPU)
float *h_P = (float*)malloc(size * sizeof(float)); // output tasveer (CPU)
float h_M[MASK_WIDTH * MASK_WIDTH]; // mask (CPU par banao)
for (int i = 0; i < size; i++) h_N[i] = rand() % 256; // 0..255 gray values
for (int i = 0; i < MASK_WIDTH*MASK_WIDTH; i++) h_M[i] = 1.0f/25.0f; // blur mask: sab 1/25
float *d_N, *d_P;
cudaMalloc(&d_N, size * sizeof(float));
cudaMalloc(&d_P, size * sizeof(float));
cudaMemcpy(d_N, h_N, size * sizeof(float), cudaMemcpyHostToDevice);
cudaMemcpyToSymbol(d_M, h_M, MASK_WIDTH*MASK_WIDTH*sizeof(float)); // mask constant memory mein bhejo — ye line khaas hai!
dim3 block(16, 16); // block = 16x16 = 256 threads
dim3 grid((WIDTH + 15) / 16, (HEIGHT + 15) / 16); // grid = 32x32 blocks
Convolution2D_Kernel<<<grid, block>>>(d_N, d_P, WIDTH, HEIGHT);
cudaMemcpy(h_P, d_P, size * sizeof(float), cudaMemcpyDeviceToHost);
printf("Convolution done. Centre pixel = %f\n", h_P[(HEIGHT/2)*WIDTH + WIDTH/2]);
cudaFree(d_N); cudaFree(d_P); free(h_N); free(h_P);
return 0;
}Manual ke printed kernel mein r = i + m likha hai (radius minus kiye baghair) — wo mask ko pixel ke neeche-daaein se shuru karta hai. Upar wale code mein hum ne centre-wala sahi tareeqa (r = row + m - radius) likha hai jo Mam ne lecture mein samjhaya hota hai. Exam mein agar manual wala exact code dekhna allowed hai to dono mein se jo lecture se match kare wohi chunna — faraq sirf in 2 lines ka hai. Boundary ke liye manual 'skip boundary pixels' bhi kehta hai, jabke ye code zero-padding karta hai — dono qabool tareeqay hain, ek chun kar consistent rehna.
Manual ka home task: ek AISI matrix jo square nahi (masalan 3 rows x 4 columns) ko ek vector (4 numbers) se guna karna. Jawab mein 3 numbers aayenge (har row ka ek). CPU aur GPU dono diye ja rahe hain.
// CPU VERSION (C) — har row ka dot product us row ke vector ke sath
// result[row] = M[row][0]*x[0] + M[row][1]*x[1] + M[row][2]*x[2] + M[row][3]*x[3]
// Misal: M = {{1,2,3,4},{5,6,7,8},{9,10,11,12}}, x = {1,1,1,1}
// Jawab hona chahiye: {10, 26, 42} (agar x={1,0,0,0} ho to jawab pehla column {1,5,9})
for (int row = 0; row < ROWS; row++) { // har row ke liye
int sum = 0; // is row ki gulak
for (int col = 0; col < COLS; col++) // row ke har number ke liye
sum += hM[row * COLS + col] * hX[col]; // guna karke jama (2D->1D: row*COLS+col)
hY[row] = sum; // is row ka jawab
}
// GPU VERSION (CUDA) — har ROW ke liye ek thread, andar wala loop wohi
__global__ void matVecKernel(int *dM, int *dX, int *dY, int rows, int cols) {
int row = blockIdx.x * blockDim.x + threadIdx.x; // mera row number
if (row >= rows) return; // guard
int sum = 0;
for (int col = 0; col < cols; col++)
sum += dM[row * cols + col] * dX[col]; // wohi guna-jama
dY[row] = sum;
}
// Launch: matVecKernel<<<1, ROWS>>>(dM, dX, dY, ROWS, COLS);Ye tumhari sab se latest file hai (20 number ki assignment) aur Lab 04/05 ko Lab 02 se jorti hai. Is mein wohi smoothing / thresholding / histogram hai, magar N = 60 ki jagah N = 2^20 (10,48,576) aur N = 2^24 (1,67,77,216) hai. N bara hone se 3 nayi cheezein aati hain: (1) waqt naapna (timing), (2) ek se zyada blocks, is liye GLOBAL index, (3) histogram mein race condition itni bari ho jati hai ke gintee saaf ghalat aati hai — phir us ke 2 ilaaj (atomic, phir shared memory).
#include <stdio.h>
#include <stdlib.h>
#include <time.h> // clock() ke liye
// teenon kaam functions mein — taake timing alag alag naap sako (Lab 04 wala hi hisaab)
void smoothCPU(int *hA, int *hC, int N) {
hC[0] = hA[0]; hC[N-1] = hA[N-1]; // kinare copy
for (int i = 1; i < N-1; i++) hC[i] = (hA[i-1] + hA[i] + hA[i+1]) / 3;
}
void threshCPU(int *hA, int *hC, int N) {
for (int i = 0; i < N; i++) {
if (hA[i] < 32) hC[i] = 0; else if (hA[i] < 55) hC[i] = 1;
else if (hA[i] < 77) hC[i] = 2; else hC[i] = 3;
}
}
void histCPU(int *hA, int *hB, int N) { // hB ek array hai: hB[0..3], shuru mein sab 0
for (int i = 0; i < N; i++) {
if (hA[i] < 32) hB[0]++; else if (hA[i] < 55) hB[1]++;
else if (hA[i] < 77) hB[2]++; else hB[3]++;
}
}
int main() {
int N = 1 << 20; // 2^20 = 1048576 (2^24 ke liye 1 << 24)
int *hA = (int*)malloc(N * sizeof(int)); // bara N stack par nahi, malloc se lo
int *hC = (int*)malloc(N * sizeof(int));
int hB[4] = {0,0,0,0};
srand(time(NULL));
for (int i = 0; i < N; i++) hA[i] = rand() % 90 + 10;
clock_t start = clock(); // ghari shuru
smoothCPU(hA, hC, N); // kaam karo
clock_t end = clock(); // ghari band
double ms = (double)(end - start) * 1000.0 / CLOCKS_PER_SEC; // milli-second mein waqt
printf("Smoothing N=%d time = %.2f ms\n", N, ms); // isi tarah thresh/hist ki timing bhi lo
free(hA); free(hC);
return 0;
}Kernel Lab 05 walay hi hain, sirf 2 tabdeeliyan: (1) i nikalne ke liye GLOBAL formula, (2) launch ke waqt threads = 256 aur blocks = (N + 255) / 256. Chote N=60 par test ke liye threads = 8 diye gaye hain assignment mein.
int firstMismatch = -1; // -1 matlab abhi tak sab barabar
for (int i = 0; i < N; i++) {
if (hC_cpu[i] != hC_gpu[i]) { firstMismatch = i; break; } // pehla farq mila, ruk jao
}
if (firstMismatch == -1) printf("matches CPU\n");
else printf("Mismatch at index %d: CPU=%d GPU=%d\n", firstMismatch, hC_cpu[firstMismatch], hC_gpu[firstMismatch]);// Pehle har thread ke liye bin nikalne ka mushtarika hisaab:
// int val = dA[i]; int bin = (val < 32) ? 0 : (val < 55) ? 1 : (val < 77) ? 2 : 3;
// VERSION (a): BINA synchronization — GALAT gintee (race condition ka saboot)
__global__ void histNaive(int *dA, int *dB, int N) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= N) return;
int val = dA[i];
int bin = (val < 32) ? 0 : (val < 55) ? 1 : (val < 77) ? 2 : 3;
dB[bin]++; // KHATRA: hazaron threads ek sath ++ kar rahe hain — gintee zaya hogi
}
// VERSION (b): atomicAdd global bins par — SAHI, magar sust (har thread line mein)
__global__ void histAtomic(int *dA, int *dB, int N) {
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i >= N) return;
int val = dA[i];
int bin = (val < 32) ? 0 : (val < 55) ? 1 : (val < 77) ? 2 : 3;
atomicAdd(&dB[bin], 1); // ek ek karke — sahi gintee, magar N dafa line
}
// VERSION (c): SHARED memory pehle, phir sirf 4 atomic per block — SAHI aur TEZ
__global__ void histShared(int *dA, int *dB, int N) {
__shared__ int sB[4]; // is BLOCK ki apni 4-dabbon wali copy
if (threadIdx.x < 4) sB[threadIdx.x] = 0; // pehle 4 threads har dabba 0 kar dein
__syncthreads(); // RUKO — sab dabbe 0 ho jayein pehle
int i = blockIdx.x * blockDim.x + threadIdx.x;
if (i < N) {
int val = dA[i];
int bin = (val < 32) ? 0 : (val < 55) ? 1 : (val < 77) ? 2 : 3;
atomicAdd(&sB[bin], 1); // shared par atomic (tez, global wali line nahi)
}
__syncthreads(); // RUKO — block ki gintee poori ho jaye
if (threadIdx.x < 4) atomicAdd(&dB[threadIdx.x], sB[threadIdx.x]); // har block sirf 4 global atomic karega
}| Cheez | Hisaab | Jawab |
|---|---|---|
| Kul threads (N) | 2^24 | 16,777,216 |
| Blocks (256 threads/block) | 16,777,216 / 256 | 65,536 blocks |
| Version (b) global atomics | har thread 1 atomic = N | 16,777,216 |
| Version (c) global atomics | har block sirf 4 atomic = 65,536 x 4 | 262,144 |
| Kitni dafa kam? | 16,777,216 / 262,144 | 64 dafa kam |
| Agar Mam Kahein... | To Sirf Ye Badlo |
|---|---|
| N = 60 ki jagah aur size | #define N wali line / N variable ki value — aur kahin kuch nahi |
| k ki nayi value | #define K 12 ki value |
| Operation badlein (+ - *) | hisaab wali EK line mein nishan (+, -, *) badlo |
| Numbers ki range badlein | rand() % (high-low+1) + low — Template A |
| Sirf even / odd numbers | loop ke andar if (hA[i] % 2 == 0) lagao |
| Block size badlein (CUDA) | threads variable ki value; blocks = (N + threads - 1) / threads |
| Bara N (ek block se zyada) | kernel ki pehli line: i = blockIdx.x * blockDim.x + threadIdx.x + guard if (i < N) |
| Bins / groups badlein | sirf < 32, < 55, < 77 wale numbers — 'agli range ki pehli value' |
| Semaphore mein ek waqt mein X threads | sem_init ka AAKHRI number X kar do |
| Mutex increment -> decrement | shared_var++ -> shared_var--; shuruati value ulta kar do |
| Ping/Pong -> Numbers/Letters | sirf cout wali lines: i+1 aur (char)('A'+i), loop 10 kar do |
| Mask 5x5 -> 3x3 (Convolution) | MASK_WIDTH 3 + mask values 1/9 (blur ke liye) |
| Image size badlein | WIDTH/HEIGHT ki values; grid formula khud sambhal lega |
| Histogram tez karna ho | shared-memory version (c): pehle block ke andar gintee, phir 4 atomic per block |
ALLAH HAFIZ WALA RULE: Exam mein naya sawal dekh kar ghabrana nahi — pehle ye dhoondo ke 'ye kis lab ke kis task jaisa hai?' Phir us task ka code likho, aur aakhir mein is Master Cheat table se sirf woh EK tabdeeli lagao jo sawal ne mangi hai. Slightly modified sawal ka jawab = purana code + ek choti si tabdeeli. Yahi is poori guide ka khulasa hai. Best of luck!
Lab 01
Element-wise add, subtract and multiply, scalar with k = 12, dot product, and sum, max, min and average for N = 60.
Lab 02
The same four tasks as GPU kernels, with atomicAdd and shared memory.
Lab 03
Mutex race condition, ping-pong with a condition variable, and semaphore.
Lab 04
Smoothing, thresholding and histogram, explained step by step.
Lab 05
The same three tasks on the GPU, including an atomic histogram.
Lab 06
512 by 512 image with a 5 by 5 mask and constant memory, plus the matrix-vector home task.
Assignment
CPU timing for large N, global indexing, and three histogram versions: race, atomic and shared memory.
This guide was prepared by Uzair Malik, a BCS Computer Science student at COMSATS University Abbottabad and founder of Zavia, from his own pre-performed labs for CSC334. It is a student study guide, not official course material, and it is not endorsed by the university or the course instructor.
The guide starts with five golden templates that repeat in almost every lab — random numbers in a range, the five CUDA steps, thread indexing, atomicAdd, and shared memory with synchronization — then works through each lab in turn, ending with a one-page master cheat table and a list of common mistakes.
Download the PDF — free for subscribers
Reading online above is free for everyone. To download the PDF, subscribe below — one email a week, unsubscribe anytime.