A non-contact vision-based system is presented for continuous respiratory rate monitoring. The system identifies feature points in a video feed and tracks them over time. Two methods are presented for comparison – a method which uses principal component analysis (PCA) and a simple averaging approach. These methods condense the feature point trajectories into a compact set of representative signals. The signal which most closely resembles an expected respiratory trace is selected based on spectral analysis. System performance is assessed by comparing the estimated respiratory rate to the rate determined via inductance plethysmogram. The system was evaluated on 5 participants in 4 simulated sleep scenarios. Accuracies of within 1 breath/minute were achieved for more than 97% of the recorded time in all scenarios. The proposed system is accurate, cost-effective, and simple, making it a suitable candidate for at-home installation.