This paper covers a novel learning algorithm fo r a HMM (Hidden Markov Model) based on PSO (Particle Swarm Optimization) is addressed, and a new video action recognition approach is proposed. Firstly, we extract the features of the action's trajectories from each of the target objects. Then, we compare them with the semantic event probability produced in the HMM. Secondly, we improve the HMM by modifying its learning algorithm parameters based on PSO. This has the advantage of changing the computational learning level of the HMM from a resu lt that is locally optimal to one that is globally optimal; Meanwhile, it can avoid the common computa tional errors associated with data overflow. Finally, we identify the objective activity patterns using the Time Wa rping Method by matching event probability sequence. Practical data experiments show that the pres ented algorithm is efficient by reflecting the real activities and can influence how the problem is attacked. The comparative experiments also show that it has advantages over the Baum-Welch algorithm and other famous methods.