The performance of most current open-domain dialog systems is limited by the (training) dialog corpora due to either generation-based or retrieval-based learning patterns. To circumvent this limitation, we propose PARL, an open-domain dialog system framework using Prompts as Actions for Reinforcement Learning. This framework requires a (fixed) open-domain dialog system as the backbone and trains a behavior policy using reinforcement learning to guide the backbone system to respond appropriately with respect to a given conversation. The action space is defined as a finite set of behaviors in the form of natural language prompts. Preliminary results show that with the guidance of the behavior policy, the backbone system could generate more engaging and empathetic responses.