With the increasing demand for scientific and military operations in marine environments, unmanned underwater vehicles (UUVs) have become essential due to their adaptability in conducting various maritime missions. Autonomous control systems further enhance this flexibility, particularly in mission-specific operations. However, much of the existing research focuses predominantly on 2D environments, often neglecting the complexities and orientation challenges that arise in real-world 3D underwater scenarios. To address these issues, this paper introduces a 5-degree-of-freedom (DOF) control approach for UUVs, aimed at improving waypoint-based path planning in 3D maritime missions through directional policy optimization (DPO). Utilizing a deep reinforcement learning (DRL) framework, the proposed method employs an efficient directional policy that accounts for the unique maneuvering characteristics of UUVs, including the maximum feasible rotation angles under dynamic constraints. The DPO algorithm enables UUVs to achieve efficient path planning by minimizing the number of waypoints required. Additionally, by considering the impact angle, which dictates the approach angle to the target, the methodology facilitates a mission execution strategy that minimizes the UUV’s exposure, thus enhancing stealth during military operations.