Category: Research Methods

#AcWri, #LTEC6040, #phdchat, Research, Research Methods

Search and Organize: The Literature Review

Laura Pasquini

As you launch into a new research project, it is critical to think about how you will SEARCH and STORE an empirical literature search. Much of this organization starts with identifying a well-defined topic of study, and then identifying literature and scholarship around the previous work in this area. This process should support how you define your research topic scope/focus (e.g. inclusions/exclusions), established research methods (e.g.data collection/analysis), existing results/findings, and potential research suggestions for further investigations.

Step 1. Identify and Select a Research Topic to Study

For my #LTEC6040 early career scholars, the focus of study is geared towards digital learning/teaching. To prepare for a literature search, I suggest a few preliminary steps to SELECT and IDENTIFY a specific TOPIC for their research projects:

  • Interest: Find something you want to know more about so you remain interested and engaged in the project investigation. This might be related to your research agenda, either for your thesis/dissertation or general interests of study.
  • Ideas: Browse current periodicals (e.g. The Atlantic, Education: NPR, New York Times, Time), and online news resources for trends in the field you are looking at for digital teaching/learning (e.g. Inside Higher Ed, The Chronicle, Ed Surge, etc.).
  • Scope: Is your topic manageable? Avoid choosing a research topic that is too broad (too much information) or too narrow (too specialized/new/limited in appeal to find enough information). Consider limiting the time span, identifying a specific digital learning setting, or limiting this to a particular sample population/course/instructional method/lens.
  • Time: Choose a research topic you can work on for a set period of time. For example, this semester (3-4 months) is the length of time to set up a small-scale study, complete the ethics review, recruit a sample population, and start on a draft of a journal article. Always plan enough time to go through the empirical literature as you prepare the investigation and draft the paper.
  • Approach: There are different approaches possible for each research topic. Scholarly papers can analyze or explain a concept, narrate events, design, or developments in the field, or even argue for or against theory or idea. Additionally, you might choose to focus on a philosophical, historical, sociological, psychological, scientific, etc. approach.
  • Perspective:  Research topics can be examined through a variety of scholarly perspectives. Each research perspective or lens requires different sources of information so it is important to establish what aspect of the topic interests you most from the start.
  • Clarity: Be clear about the topic you are researching. Your research topic might need some adjustment as you gather information; however, you should always have a well-defined focus for your topic of search to ensure you stay on track and avoid wasting time with your literature search.

Step 2. Find Scholarly and Peer-Reviewed Evidence

Once this RESEARCH TOPIC is identified, next step is the literature review. This is critical part of research helps you to identify scholarly publications with evidence and investigation processes for your topic. Hart (1998) believes the literature review is an evaluative process is to determine how this empirical publications to answer the  following QUESTIONS for your scholarly search:

  • What are the key sources?
  • What are the major issues and debates around the topic?
  • What are the key theories, concepts, and ideas?
  • What are the epistemological and ontological grounds fro the discipline?
  • What are the political standpoints?
  • What are the origins of this topic?
  • What are the definitions involved with this topic?
  • How is knowledge on the topic structured and organized?
  • How have approaches to these questions increase our understanding and knowledge?

Step 3. Limit and Refine the Empirical Search

To help you keep your focus and direction for the literature search, it is critical to have a definitive argument/focus for your study. To best evaluate the scholarly works, Belcher (2009) offers suggestions to refine your literature review by reading materials that contribute to the central argument of your manuscript and limiting the following items for your search:

  • Set a time limit:  i.e. read nothing written over 10 years ago, five, or two (depending on your topic of research)
  • Language: limit to articles in English (or designated languages/preferences)
  • Questionable publishing outlets e.g. trade journals, non-peer reviewed, some conference proceedings not always suitable
  • Different geographical areas (by author country of origin)
  • Different time periods (related to your genre — this might apply to humanities more)
  • Different kinds of experiments (by your methods of study/research)
  • Different kinds of participants (by research sample type, size, etc)
  • Different variables (e.g. gender, age, etc.)
  • Without your keywords in the title or abstract – focus your search for these items
  • Non-electronic formats – if you can’t access the research from home/library resources

Step 4. Establish a System to Organize the Research Collection

Be sure to keep track of papers collected into a system of review. This might involve storing files, taking notes/annotations, and organizing articles into a citation system. I suggest setting this up early in your research life and I would definitely encourage you to use citation management software to track, store, and annotate articles you find for your literature search. Here are two platforms I use to tame the citations and literature collections [Learn more about citation management via the Research in Action podcast, episode no. 36]:

  1. Mendeley is a combination of a desktop application and a website which helps you manage, share and discover both content and contacts in research. You can store, save, annotate, and share documents with scholarly collaborators, plus manage and sync your references with a team in a group, or for yourself. You can easily drop in PDFs into the system to tag, cite, highlight, and organize your literature review. This is an excellent tool for team research and writing projects working from a distance.
  2. Zotero is utilized by a number of scholars as it is “an easy use tool easy-to-use tool to help you collect, organize, cite, and share your research sources.” Zotero hosts research groups and individuals who want to connect and collaborate with other scholars OR discover the works of others. It contains several disciplines through which a user can keep updated on and search for people to connect with. It is free to sign up and you download it to your computer.

Step 5. Search, Track, and Locate Relevant Papers

Find a way to organize and keep track of what you are searching (terms, keywords, filters, search strings, etc.) and where you might be finding these resources. Depending on your institution or access, you might not have a way to find ALL THE LITERATURE. Here are a few ways to expand your literature expedition and get access to empirical papers beyond your reach:

  • Search, Track and Set Alerts: Record the different search strings to recall what you find and perhaps to set up an alert (e.g. in Scopus, Summons, Google Scholar, etc.) that is relevant for this research topic or methodology. Here’s a screenshot of a Google spreadsheet for search for one of my projects: Screen Shot 2016-06-19 at 1.15.32 PM
  • Google Scholar search the “Cited by ###” section of the site: this is to identify other relevant paper on topic or learn more about this research thread, i.e. a discovery search for missing literature. This will also bring about the “grey” research that is not indexed or part of a database search.
  •  Use Backward & forward referencing search method: for collecting and reviewing publications to be inclusive of empirical literature. This might bring about relevant publications to be included in your own review and give insights to your topic OR the research methodology.
  • Search for Publications Beyond Reach: Articles that you are not able to access at your institutional library or databases I have access to, I will tweet #iCanHazPDF [in action #icanhazpdf] to ask my professional network on Twitter OR even connect to the author (by email, social network, etc.) to ask for a pre-print copy. Beyond this you can find articles via other academic search engines with access, such as Sci-Hub, Semantic Scholar, and many more research databases.

Step 6. Process and Understand the Literature Gathered

Beyond these methods for storing papers, think about how you will process and organize your literature collection. You might have notes drafted as you review to identify themes, issues, and concepts you want to include for your own paper. Here are few tips/tricks I’ve honed as I search for literature:

  • Take Fewer Notes: Tag articles in the software,  group articles into specific folders, skim abstracts to code/organize, and identify literature for easy recall and use later. Have a system for your own tags or references to recall/use later when writing. Make meaningful labels that connect to your specific research focus/scope.
  • Create an Annotated Bibliography: For smaller literature searches (or team support efforts) start a reference list of your citations in APA 6th Edition format with a brief line or two making about the study, methods, findings + personal thoughts on articles/methods you read for each citation. Create annotations on this reference list, these are notes, on why this paper is relevant and could be helpful for your own research.
  • Concept Mapping the Literature: It might help you to create a visual or graphic organizer to map out these ideas and piece items together for your own manuscript. This might be digital or even analog — break out the markers, crayons, post-its and more! Check out the great suggestions Pat Thomson offers on “spaces between the literature” for reviewing research; a.k.a. bushwhacking
  • Don’t Wait to Write: From the annotations and notes you have made on the collected papers, start organizing how and where they might fit into your article draft. These preliminary notes might be rough; however, if you pop some of your literature into an outline of a paper this will help you write a draft of an article when you have your data collected and analyzed.

What other advice do you have for getting started on a new research project? What suggestions do you have for searching and organizing a literature review? Let me know — I’m always keen to get a few new ideas for scholarship and practice. Thanks!

References:

Belcher, W. L. (2009). Writing your journal article in 12 weeks: A guide to academic publishing success. Thousand Oaks, CA: Sage Publications Inc.

Hart, C. (1998). Doing a literature review: Releasing the social science research imagination. Sage.

Podcast, Research Methods

Research, Interviews, and Asking Good Questions

Laura Pasquini

I have been thinking about interviews and how to ask better questions/interview for a while. Research questions unpack what is going on with the world around us. As an early career scholar, I want to unpack experiences, thoughts, and situations people are dealing with in the workplace (e.g.  networked professional lives, open online learning, mentoring relationships) to learn more about a particular phenomenon. I know good research comes from solid research preparation.

Last summer  I spent a couple of months, with my co-investigator Paul, digging into the empirical literature, academic findings, theoretical frameworks and debates around concepts and issues we want to unpack in our study. I appreciate his willingness to work and put the time up-front to prepare for our research interviews.

“Research designs begins with questions researchers and their partners want to answer about a particular problem, population, process, project, or topic they want to explore” (LeCompte & Schensul, 2010, p. 130).

We framed our research questions around issues addressed in other academic papers — you know, building on the shoulders of giants — and to unpack what is happening in the online and offline realm for higher education professionals. For our semi-structured interviews, we have a set of structured questions to guide open-ended discussions on relevant topics related to the themes, issues, and concepts we want to discuss (Kvale, 2007). By using the intensive interview techniques shared in Charmaz’s (2014) constructing grounded theory text, most of our questions are open-ended. This method was designed to encourage participants to reflect and share experiences, by starting questions with: “Tell me about…”, “Could you describe… or “Can you walk me through…”  Asking research questions to solicit for a comprehensive and an open response is everything.

This research design thinking not only developed our interview protocols, research questions, and data management plan, it also allows us to be fully immersed in our conversations while we conduct the interview now.  I think conducting a quality research interview is a skill. A skill that gets developed, honed and enhanced as you go. I always learn how to improve upon this each time I talk with a research participant. While being immersed in the interviews, I have kept this sage advice George (thanks!) offered when we were conducting interviews with a large number of open, online learners:

  • Give wait time to think before answering and tell them that you are doing that.

  • Listen to their replies and ask probing questions that aren’t listed below but go toward the issues we are trying to explore.

Now that we’re 60+ interviews deep with our project, I continue to think about this advice and understand what we are learning so far. I am also thinking about what we are asking, how we are approaching topics, and identifying where we might need to go as our questions reach a certain saturation point. If you have already graciously volunteered your time and shared for our study: THANK YOU SO MUCH!  If you are a higher education professional who would like to contribute and be interviewed for our research, we are still accepting participants for our study here: http://bit.ly/networkedself

UPDATED: Friday, August 11, 2017

R.I.P. #Turnaroundpod — it’s sad to hear that your podcast series is coming to a close. THANKS SO MUCH for producing The Turnaround Podcast, Jesse. It will be sad to see you go! Want to read more about this? Check out the Ask Me Anything (AMA) of Jesse Thorn on Reddit.

Recently, I started listening to Jesse Thorn’s  The Turnaround podcast (that partners with the Columbia Journalism Review -thanks for the transcripts!) This podcast flips the script and interviews people who typically interview others.

the-turnaround-cover_6

Image c/o The Turnaround! a Maximum Fun Production

These interviews unpack the art form of an interview and how to best investigate a story. Thorn asks how to best interview and also demonstrates how to summarize ideas and follow with an open-ended question for a response. Although most of these interviewers are producing interviews for public consumption and listening, there are some great takeaways from this 1:1 series about interviewing:

In addition to listening to podcasts or reading scholarly books about interviews, I thank and credit the @BreakDrink podcast production for providing me with the skills to conduct effective research. My “study” in podcasting (and research interviews) began just over 7 years when I received a DM from Jeff Jackson to see if I’d like to co-host a podcast. Although I was just starting my Ph.D. program, I think some of my early lessons for qualitative research actually came from the episodes where we invited brilliant people onto the Campus Tech Connection (#CTCX) podcast for an interview. Both my experience with podcast production and research interviews, have offered me a few insights for being a more effective interviewer:

  • Pre-Interview survey: Ask your podcast guest or interview participant a few questions about the topic in advance. For podcasts, we would have them complete a brief bio and see a few of the questions we might ask ahead of time. For interviews, we might have a pre-questionnaire or interview sign-up with requests for demographic information, topics about the research, or their role for the study research.
  • Organize and prepare: Do you work in advance! Create a shared doc (if on a collaborative team) or prep notes for each show production or segment of your research interviews. This would include the potential protocols, research questions, interview topics/issues, and information you would need for each recording. Review the pre-interview survey data and see how they might relate to your research questions.
  • Play with the technology to figure out what works for you: Technical tools have changed over the past 7 years of my podcasting/researching. I continue to learn as I go and as I collaborate with others. I now record with Audio Hijack+Skype/web conference/phone, edit in GarageBand/Audacity by splicing clips either for public consumption or to minimize for transcription costs, and find a secure cloud storage space for your audio files and notes.
  • Speaking of notes… ALWAYS TAKE NOTES: Besides recording the audio, I often scribe notes during a conversation or interview. These notes could include a quote, key point, idea, or issue. For the podcast, this might include a URLs and resources we would share with the show notes with the episode. For research, this ensured I was listening and noting what participants were saying and often it would spark a follow-up question or explore another aspect of our study I wanted to know about.  Pro-Tip: I use “analog” journals to write my research notes with pen and paper. I often return to my notes to make an annotation, highlight a concept, find another research question, and to review how the series of interviews are progressing.
  • Make time for reflection: After each episode of the podcast, I often would have a follow-up blog post with information and ideas shared. This practice I still do when I conduct a research interview, but often it’s a private act scribed in my journal or shared with my co-collaborators on a project.  This habit has me process what I am exploring, learning, and sorting out in my head.
  • Manage and archive your files: Be sure you create a system to label and itemize your digital files and notes. I am meticulous for organizing my life and projects (as I live in the digital) in particular ways. Set your own system so you can track where items are and code how these files/interviews are relevant to your project (or podcast). This will help you later when you go to code transcripts or you are interested in a particular issue/trend in your study.

References:

Charmaz, K. (2014). Constructing grounded theory.Thousand Oaks, CA: Sage.

Kvale, S. (2007). Doing interviews. Thousand Oaks, CA: Sage.

LeCompte, M. D., & Schensul, J. J. (1999). Designing and conducting ethnographic research (Vol. 1), 2nd Edition. Plymouth, UK: Altamira Press.

Fashioning Circuits, Research Methods

Participation Observation Method

Laura Pasquini

In constructing the curriculum chapter for the Fashioning Circuit book* being developed by Dr. Kim Knight (a.k.a. @purplekimchi), I utilizing a few exploratory research methods to review the current workshop materials, lessons, and learning on the subject matter. The first method: Participant Observation. As I work through evaluation and assessment of the curriculum, I might as well share and get feedback on the process.

EFC Camp

Participation observation allows for the collection of information and qualitative data, rooted in the ethnographic research tradition. For this method, participation observers report on the physical, social, and cultural context to reveal relationships, activities, and behaviors of subjects. This is an effective method to gather information to support project design, data collection development, and to interpret other research. Data collection for this method includes note-taking, mapping-relationships, and media (video, audio or images) that might be translated into textual artifacts. Challenges to this method include diligent documentation and objective account from observers in the field, and this process can be time-consuming.

Specific responsibilities for Participant Observers include:

  • observing individuals as they engage in activities (as if you were not present and watching)
  • engaging in the activities to gain a better understanding
  • interacting in a controlled research environment
  • identifying and developing relationships with key informants and stakeholders

For the purpose of this research, I developed a field guide for our research team of three. Basics for the observation guide include listing the observer name/background, research setting, materials used, and concentration areas to focus on for the workshop observation. Other tips and general guidelines were provided to outline expectations for observing.

The research team divided and conquered today by taking notes related to the following categories:

  1. Lesson/Curriculum (Electronic Fashioning Circuits Camp)
  2. Lead Instructor/Facilitator (a.k.a. Dr. Knight)
  3. Learners/Students (participants in the workshop)
  4. Facilitators/Helpers (those supporting the workshop)

The observation guides were segmented by the 4 categories and included questions to prompt observers and focus their field notes.  The observation goal was to focus on the physical space and set up, participant attributes and involvement, verbal behavior and interactions, physical gestures, personal space, lesson understanding, instructional support, and individuals or examples that stood out from the workshop.

At the beginning of the day our group met to review the research context, expectations, behavior as an observer, and potential problems that might occur during the workshop. Another item we discussed was distinguishing interpretation (I) from observation (O), and labeling our notes accordingly (Kawulich, 2005). To help with strategic note-taking, I encouraged leaving space to expand on notes, using shorthand to follow up with later, writing observations in  sections, and encouraged our team of researchers to consider body language, attitudes, conversations, ambiance, and general interactions that might be relevant for the curriculum.

Participant Observatin Continuums

Image c/o Chapter 3: Participation Observation (Guest, Namey, & Mitchell, 2012)

During the day the three of us took notes on tablets, laptops, mobile phones, and pads of paper with the following platforms: Google docs, Word, Evernote (audio & images), etc. We reconvened the end of the workshop to process and discuss what we observed. This debriefing provided ideas for supporting a research team, specifically with regards to:

  • general observations, ideas, and questions about the workshop
  • how to create anonymous identifiers for research subjects in notes
  • expectations for field note-taking and organization submission for the lead researcher
  • roles and responsibility for how to effectively observe a single group within in a workshop, i.e. instructor, learners, and helpers
  • future planning needs and ideas for upcoming participation observation

I am truly grateful for the UT Dallas EMAC students, Jodi & Lari, who volunteered their time to observe and be a part this exploratory study. Their insights and ideas are very helpful for future field observations and research method development. Once everyone’s participation observation notes and artifacts are collected, I will share how to analyze this data.

Lily pad

*Interested in learning more about Fashioning Circuits? There’s a few social spaces for that! Check out the Fashioning Circuit’s website, Facebook page, Twitter handle or hashtag #FashioningCircuits. Feel free to follow along, and join the conversation.

 

References:

Guest, G., Namey, E. E., & Mitchell, M. L. (2012). Collecting qualitative data: A field manual for applied research. Sage.

Kawulich, B. B. (2005). Participant observation as a data collection method. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 6(2), Art. 43, http://nbn-resolving.de/urn:nbn:de:0114-fqs0502430.

Research Methods

Action Research Methods

Laura Pasquini

Action research (Koshy, 2005), also known as participatory action research, is a method of research that can combine a framework for public, reflective inquiry.  It offers a way for academics and scholars to work with individuals or communities, such as learners or communities of practice, to investigate issues together to find a solution-oriented approach. For educational settings, this might often be the “teacher-as-researcher” research approach (Elliot, 1991; McNiff, 2013).

action_research

Image c/o New Mexico State University, College of Education Research and Budgeting

The premise for the action research method is that  researchers study with the subject(s), rather than research on or outside the subject(s) as a hierarchy or “expert” (Cousin, 2008). Action research is a journey that experiences transformative findings with linear, sequential relationship between the hypothesis, research activity and the research findings, i.e. a proposed solution.

Another branch of this research includes, community-based participatory action research (CBPR), involves researching a group or community where research is conducted:

“by community members, so that research results both come from and go directly back to the people who need them most and can make the best use of them.“

CBPR research will grow in education as it is impacted with emerging technologies and learning pedagogy design, while needing to meet standards and managing fiscal challenges. This research method allows for project ideas and measurement outcomes to be designed by the community population to shape the research agenda. Participation in research by the community often results in generating greater sociopolitical awareness and effecting systemic change in the community (Jason et al., 2004). From  other case studies described by Jason et al. (2004), CPBR has improved upon the quality of scholarly work, developed research outcomes, created intervention tools, and increased quality collaborations on projects.

References

Cousin, G. (2008). Researching learning in higher education: An introduction to contemporary methods and approaches. Routledge.

Elliott, J. (1991). Action research for educational change (Vol. 49). Buckingham: Open University Press.

Jason, L. A., Keys, C. B., Suarez-Balcazar, Y. E., Taylor, R. R., & Davis, M. I. (2004). Participatory community research: Theories and methods in action. American Psychological Association.

Koshy, V. (2005). Action research for improving practice: A practical guide. Sage.

McNiff, J. (2013). Action research: Principles and practice. Routledge.

 

Fashioning Circuits, Research Methods

Ethnographic Methods

Laura Pasquini

In thinking about how to evaluate the curriculum and pedagogy for the #FashioningCircuits book with Dr. Kim Knight (@purplekimchi), I spent some time over the weekend reviewing  potential research methods for the project. Since I have used  ethnographic methods for a grant project Fall 2012 (studying culture and healthcare perceptions), I thought I’d revisit these as potential research methods and blog about it here.

Ethnographic Methods

Ethnographic is the description of groups, specifically these approaches appeal to those who are confident they have time to stay in a research setting (the field) for a sustained observation and informal interviewing (Cousin, 2008).

ethno

Typically ethnographic research derives from anthropology where communities or groups are studying for long periods of time (a year or so) for the “purpose of learning from their ways of doing things and viewing reality” (Agar, 1980, p. 6) where the researcher not only observes, but also interacts and talks with participants (Delamont, 2002, p.g. 8). Other data collection can be collected with communities also interact in online spaces and contexts. Ethnographers can study forums, wikis, blogs, or Twitter in the context of participant observation research to review the community and activity or historical research archive (Boellstorff et al., 2012).

A few selected strategies for research inquiry:

  1. Observation: Researchers will complete overt participant and observer review strategies which the ethnographer has negotiated access to the group to observe and participate in its activities (Cousin, 2008). Observations of this group will include notations about the environment, population, interactive order, and characteristics of the membership. In thinking about digital footprints, historical archives and other means of data collection for “observation” of this group is inclusive of blog posts, videos, tweets, and other online contexts (Boellstorff et al., 2012).

  2. Semi-Structured Interviews using the Ethnographic method (Schensul et al., 1999) provides researchers with a structured set of themes to organize the interview discussion. Unlike the structured interview, interviewers will be required to adapt, modify, and augment the prepared questions if they flow of the interview discussion suggests it (Cousin, 2008). The interviews discussions are “meaning making events” where the ethnographer utilizes “active interviewing” skills (Holstein & Gubrium, 1997). From a common list of questions, ethnographers will interview various members of the group. These interviews will be transcribed and evaluated for thematic information and content using a grounded approach.

References

Boellstorff, T., & Marcus, G. E. (2012). Ethnography and Virtual Worlds: A Handbook of Method. Princeton University Press.

Cousin, G., Cousin, J., & Deepwell, F. (2005). Discovery through dialogue and appreciative inquiry: A participative evaluation framework for project development. D. Taylor and S. Balloch (Eds.), The politics of evaluation, 109-118.

Cousin, G. (2008). Researching learning in higher education: An introduction to contemporary methods and approaches. Routledge.

Delamont, S. (2002). Fieldwork in educational settings: Methods, pitfalls and perspectives. Routledge.

Holstein, J. A., & Gubrium, J. F. (1997). Active interviewing (pp. pp-113). Sage Publications.

Schensul, S. L., Schensul, J. J., & LeCompte, M. (1999). Essential ethnographic methods. Walnut Creek.