EvalAnswer is a tool for automatically detecting different types of nonresponse in answers to openended questions. It was specifically developed for use in web probing procedures but it can be used in any online survey that asks open-ended questions. EvalAnswer automatically detects and codes cases of nonresponse and suggests follow-up questions which are tailored to reduce nonresponse. Once implemented in a survey, researchers have a powerful survey aid that helps to automatically increase data quality during the interview process by eliciting better answers to open-ended questions. Furthermore, the tool can be used in the post-processing of answers after data collection. The tool can be adapted to be used in ones own survey. This paper describes the survey methodology that led to developing the tool and it gives details on its validity and effectiveness.