The Essentials of Survey Study and Reporting

We frequently receive requests on our smartphones to respond to surveys on opinions, attitudes, and practices related to our profession. Survey studies account for an increasing proportion of published research.[1] With the widespread use of the Internet and smartphones, the past two decades have witnessed a massive rise in these studies, particularly since the COVID-19 pandemic. It is relatively easy to get quick data, cost-effective, and much easier to conduct and analyze with user-friendly web tools. They are ideal for measuring elements that are not directly observable, such as attitudes, opinions, and beliefs, including satisfaction and feedback. Well-constructed and validated survey studies can provide vital and reliable insights, sometimes the only tool for some research questions.

Online surveys are the simplest form of research, gaining popularity among trainees and novice researchers. Unfortunately, many such surveys are poorly designed, resulting in incorrect inferences and biased reporting, casting questionable reputation on survey studies. Indeed, the enthusiasm to quickly conduct a survey and get published in their names is appreciated and understandable. It is, however, imperative for authors to dive deep into the principles of these studies and understand and apply them for good outcomes. It is beyond the scope of this editorial to provide detailed steps to carry out such studies but encourage and assist authors in following all essential components of the research and include them in the manuscript preparation. By adhering to fundamental principles, the critical questions raised by the authors and the responses should accurately reflect reality. Studies with ineffective research questions and incomplete or flawed designs are considered inadequate reporting and most likely to be rejected.

Essential Steps

There are a few key principles in survey research that should be followed.[2] They include the following:

• Clear research question: Like in any other research, the foremost step in designing the survey is to define what you are looking for, the primary research question. The survey should focus on eliciting specific essential information than a blanket approach to get as much information as possible. Keep the survey as small as possible, focused on the intended information, without redundant questions.

• Designing the study and questionnaire: Decisions regarding the format, structure number, and order of survey questions can be quite complex, and developing them from scratch is a huge task. Most novice researchers may not understand the principles of valid questionnaire design. A good literature review, referring to previous similar study formats and adopting them with suitable modification, without copyright infringement, is quite useful.

• Open versus closed question formats: Closed-ended questions provide a set of options to choose from, while open-ended questions allow responders to come up with their own answers. The former is easier to evaluate and the data are more valid statistically, while the latter is more useful when seeking expert opinions or feedback from a small group. However, the closed-ended questions need initial efforts to frame appropriate responses, while in the open-ended format, although it easier to set up initially, it is much more difficult to analyze the data. The questions should be clear, concise, and without bias, intendant to address the research question.

• Pretesting the survey instrument with pilot test: Although not mandatory, it is useful to test the completed questionnaire among a small group of target subjects so that few unconsidered problems, such as ambiguity, language errors, etc., are resolved.

• Validate the survey instrument: Establishing the validity of the survey instrument is the responsibility of the investigator. It simply means how accurate and reliable the data are to measure what is intended. Adopting already validated and published tools from peer-reviewed studies is rather easier. If the survey is new, establishing validity is essential based on different types. These include the following: face validity (does it make sense simply by looking at it?); content validity (a panel of experts reviews and revises it for relevant content); criterion validity (how does it compare with an established standard in the field?); predictive validity (how does it compare with the actual outcomes of interest?); and construct validity (does the internal construct/structure conform to scientific survey study principles?).

Limitations and Biases

The major limitations of survey studies are the participants' willingness to respond, and the response rate is crucial to evaluate the validity of the study.[3] There are no defined criteria for an adequate response, but a response rate of anything between 50 and 70% is considered adequate. The method of survey too plays a crucial role, with the highest response to telephonic interviews (>80%) against e-mail surveys (24%).[3] Poor response rate leads to bias in the study, such as “nonresponders bias,” calculated from the variance between responders and nonresponders. Studies with less than 20% response rate are not considered for publication.

The Manuscript Preparation

Based on the intent and construct of the study, the manuscript could be submitted as an original research paper, brief communication, or a letter with limited information. Thorough and complete details of essential steps are mandatory for peer reviewing and considerations for acceptance. These are the following[2] [4]:

• The introduction: It should describe the rationale for using the survey, how the research question will be addressed, and the target population.

• Methods: The methodology is crucial and describes whether the survey questionnaire was developed by the authors or adopted from previous studies and the details about the framing of the questions, the experts involved, and the relevance and rationale behind it. The description of whether the survey was pretested or a pilot study was undertaken before conducting the survey. The execution of the survey, including mode of contact (web-based, interview, telephone, or postal), the timing of the study, whether anonymity was maintained, how long was the response time, reminders, incentives, etc., should be mentioned in detail.

• The results: Describe who the respondents were, the response rate, and how it was calculated, and how nonresponse bias was assessed and corrected. Describe how score reliability and validity were assessed and the statistical methods used.

• Discussion: Authors construct their arguments on why their study findings should be believed and their survey is relevant to the current literature. Discuss the new insights and whether they adequately addressed the primary research question. It is essential to address all limitations and biases and refrain from overinterpretation of results.

• Data submission: Besides providing concise tabulated data, it is desirable to submit the entire survey questionnaire as an appendix or supplementary file for reviewing the manuscript and future research.

Final Thoughts

Surveys are a critical tool in educating health professionals. Compliance with the fundamental steps and guidelines of the survey studies is reemphasized here. A detailed manuscript with transparent data, validated tools, addressing biases will deter advocating an invalid hypothesis. An in-depth approach is expected from authors than merely treating surveys as desktop research.

Conflict of Interest

None declared.

• References

• 1 Artino Jr AR, Durning SJ, Sklar DP. Guidelines for reporting survey-based research submitted to academic medicine. Acad Med 2018; 93 (03) 337-340
• 2 Panacek EA. Survey-based research: general principles. Air Med J 2008; 27 (01) 14-16
• 3 Chung KC, Huynh KA, Rohrich RJ. Common fallacies in designing a research project: guidance principles. Plast Reconstr Surg 2019; 144 (05) 1247-1253
• 4 Gaur PS, Zimba O, Agarwal V, Gupta L. Reporting survey based studies - a primer for authors. J Korean Med Sci 2020; 35 (45) e398

Publication History

Article published online:
23 June 2023