Dynamic Protocol Reverse Engineering

Round trip engineering of software from source code and reverse engineering of software from binary files have both been extensively studied and the state-of-practice have documented tools and techniques. Forward engineering of protocols has also been extensively studied and there are firmly established techniques for generating correct protocols. While observation of protocol behavior for performance testing has been studied and techniques established, reverse engineering of protocol control ow from observations of protocol behavior has not received the same level of attention. State-of-practice in reverse engineering the control ow of computer network protocols is comprised of mostly ad hoc approaches. We examine state-of-practice tools and techniques used in three open source projects: Pidgin, Samba, and rdesktop. We examine techniques proposed by computational learning researchers for grammatical inference. We propose to extend the state-of-art by inferring protocol control ow using grammatical inference inspired techniques to reverse engineer automata representations from captured data ows. We present evidence that grammatical inference is applicable to the problem domain under consideration.