Merge pull request #1388 from amanver45/main

add [NEW ALGORITHM] Viterbi Algorithm for Hidden Markov Models

Miscellaneous Algorithms/Viterbi Algorithm/Program.c

@@ -0,0 +1,87 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <math.h>
+
+#define NUM_STATES 3     // Number of states
+#define NUM_OBSERVATIONS 5  // Number of observations
+
+// Function to find the maximum of two values
+double max(double a, double b) {
+    return (a > b) ? a : b;
+}
+
+// Viterbi algorithm function
+void viterbi(int observations[], int num_observations, double start_prob[], double trans_prob[NUM_STATES][NUM_STATES], double emit_prob[NUM_STATES][NUM_OBSERVATIONS], int *result_path) {
+    double v[num_observations][NUM_STATES]; // DP table to store probabilities
+    int path[num_observations][NUM_STATES]; // Path table to store backtracking
+
+    // Initialization
+    for (int i = 0; i < NUM_STATES; i++) {
+        v[0][i] = log(start_prob[i]) + log(emit_prob[i][observations[0]]);
+        path[0][i] = 0;
+    }
+
+    // Recursion
+    for (int t = 1; t < num_observations; t++) {
+        for (int j = 0; j < NUM_STATES; j++) {
+            double max_prob = -INFINITY;
+            int best_state = 0;
+
+            for (int i = 0; i < NUM_STATES; i++) {
+                double prob = v[t - 1][i] + log(trans_prob[i][j]) + log(emit_prob[j][observations[t]]);
+                if (prob > max_prob) {
+                    max_prob = prob;
+                    best_state = i;
+                }
+            }
+            v[t][j] = max_prob;
+            path[t][j] = best_state;
+        }
+    }
+
+    // Termination
+    double max_prob = -INFINITY;
+    int last_state = 0;
+    for (int i = 0; i < NUM_STATES; i++) {
+        if (v[num_observations - 1][i] > max_prob) {
+            max_prob = v[num_observations - 1][i];
+            last_state = i;
+        }
+    }
+
+    // Backtracking
+    result_path[num_observations - 1] = last_state;
+    for (int t = num_observations - 2; t >= 0; t--) {
+        result_path[t] = path[t + 1][result_path[t + 1]];
+    }
+}
+
+// Main function to test the Viterbi algorithm
+int main() {
+    int observations[NUM_OBSERVATIONS] = {0, 1, 2, 1, 0}; // Sequence of observations
+    double start_prob[NUM_STATES] = {0.6, 0.3, 0.1}; // Starting probabilities
+    double trans_prob[NUM_STATES][NUM_STATES] = {
+        {0.7, 0.2, 0.1},
+        {0.3, 0.5, 0.2},
+        {0.3, 0.3, 0.4}
+    }; // Transition probabilities
+    double emit_prob[NUM_STATES][NUM_OBSERVATIONS] = {
+        {0.5, 0.4, 0.1, 0.2, 0.3},
+        {0.1, 0.3, 0.6, 0.2, 0.5},
+        {0.2, 0.2, 0.2, 0.5, 0.3}
+    }; // Emission probabilities
+
+    int result_path[NUM_OBSERVATIONS]; // Array to store the most probable path
+
+    // Run the Viterbi algorithm
+    viterbi(observations, NUM_OBSERVATIONS, start_prob, trans_prob, emit_prob, result_path);
+
+    // Print the most probable path
+    printf("Most probable path of states: ");
+    for (int i = 0; i < NUM_OBSERVATIONS; i++) {
+        printf("%d ", result_path[i]);
+    }
+    printf("\n");
+
+    return 0;
+}

Miscellaneous Algorithms/Viterbi Algorithm/README.md

@@ -0,0 +1,52 @@
+# Viterbi Algorithm (Hidden Markov Model)
+
+This C program demonstrates the implementation of the Viterbi algorithm for finding the most probable sequence of hidden states in a Hidden Markov Model (HMM), given a sequence of observed events. The algorithm operates efficiently with a time complexity of O(T * N^2), where T is the length of the observation sequence and N is the number of states in the HMM.
+
+The Viterbi algorithm is a classical algorithm used in various fields such as natural language processing, bioinformatics, and speech recognition. It works by utilizing dynamic programming to compute the most likely sequence of hidden states that results in the observed sequence.
+
+## Key Operations:
+- **Initialization:** Sets the initial probabilities for the states based on the starting distribution and the first observation.
+- **Recursion:** Iteratively computes the maximum probabilities of reaching each state at each time step based on previous states and their transition probabilities.
+- **Backtracking:** Traces back the most probable path of hidden states once the probabilities have been computed for the entire observation sequence.
+
+## Sample Input and Output
+Consider a scenario where there are 3 states and a sequence of 5 observations. The program will calculate the most probable sequence of hidden states based on the given probabilities.
+
+**Sample Observations:**  
+`O = [0, 1, 2, 1, 0]`
+
+**Sample Probabilities:**  
+- Start probabilities: `P(start) = [0.6, 0.3, 0.1]`
+- Transition probabilities:  
+P(trans) = { {0.7, 0.2, 0.1}, {0.3, 0.5, 0.2}, {0.3, 0.3, 0.4} }
+- Emission probabilities:  
+P(emit) = { {0.5, 0.4, 0.1, 0.2, 0.3}, {0.1, 0.3, 0.6, 0.2, 0.5}, {0.2, 0.2, 0.2, 0.5, 0.3} }
+
+**Program Output:**
+
+This output indicates the most probable sequence of hidden states for the provided observations.
+
+Most probable path of states: 0 1 2 1 0
+
+## Example:
+1. **Define the Hidden Markov Model:**
+   - **States:** Three hidden states represented as {0, 1, 2}.
+   - **Observations:** A sequence of observed events.
+
+2. **Algorithm Execution:**
+   - The algorithm initializes probabilities based on the first observation, then iteratively calculates the most probable paths through the states for each observation using dynamic programming.
+   - After processing the entire observation sequence, it reports the most likely path of hidden states.
+
+## Time Complexity:
+- **Initialization:** The initialization takes **O(N)** time, where **N** is the number of states.
+- **Recursion:** For each observation, the algorithm updates the probabilities for all states, resulting in a time complexity of **O(N^2)** for each observation.
+- **Overall Complexity:** The algorithm runs in **O(T * N^2)**, making it suitable for moderate lengths of observation sequences and state spaces.
+
+## Key Features:
+- **Dynamic Programming:** The Viterbi algorithm uses dynamic programming to optimize the search for the most probable hidden states.
+- **Adaptability:** The algorithm can be adapted to various problems involving sequential data and probabilistic modeling.
+
+## Applications:
+- **Natural Language Processing:** Used in part-of-speech tagging and speech recognition.
+- **Bioinformatics:** For gene prediction and sequence alignment.
+- **Finance:** In modeling and predicting market trends based on time-series data.