In the project "Social Issues and Identity Formation in Estonia, Ukraine and Uzbekistan," researchers from a variety of disciplinary and professional backgrounds attempted to carry out research using an identical qualitative method across a number of very different settings, using six languages, and a large number of moderators. The focus group method was to be as uniform as possible in all settings, with the intention of establishing a basis for comparing results from qualitative research. In this paper I will try to assess the degree to which the goal of comparability was achieved, and where weaknesses in comparability lie. In this paper I will refer to about 24 of the 36 focus group transcripts which this project produced (8 from each country); this paper would certainly be more complete if I had taken time to read and analyze the structures of all of the transcripts, but, unfortunately, this is based on a rather quick overview which does not include every transcript. Direct references to the presence or lack of a certain question or feature in a focus group are certainly not intended as a critique of any of the colleagues in this study, but rather, and strictly, as examples to demonstrate what makes comparison among groups and across sites easier, and what makes such comparisons more difficult or impossible.

The assessment I will make concerning the comparability of the focus group transcripts includes attention to the following aspects: the degree of moderator control; the ways that the moderator shaped introductions to each section of the discussion; the ways that the moderator organized inputs and discussion of the two major portions--improvements and problems; the kinds of probes that the moderator used; and whether the moderator raised all of the questions that he or she was told to raise. This assessment is carried out with the hypothesis that the moderator has a significant degree of control of the process of eliciting ideas and opinions from participants. While I will concentrate on factors that will bear out this hypothesis, I will also bring into the analysis a consideration of volunteered information--that which the moderator did not prompt at all, but participants brought up on their own.

First, in a very basic way, the dynamics of conversation can be depicted on a scale: at one end, some focus groups were completely moderator-led; at the opposite extreme, some were free-flowing, with very little moderator input. If "m" is moderator, and "p" is participant, the conversational dynamics can be abbreviated as m/p/m/p/m/p, where the moderator introduces a topic and systematically asks for a response from each member; or m/p/p/p/p, where a moderator introduces a topic, and after an initial request for a response, other members add comments, discuss and argue among themselves; or as p/p/p/p, where a participant introduces a new topic quite apart from the moderator's agenda, and other participants comment and argue. Other recurring patterns are m/p1/m/p1/m/p1, where the moderator repeatedly questions one participant, and p1/p2/p1/p2, when two participants argue their points of view at length. All of these dynamics can be found in these focus group transcripts, and in some cases, all in the same transcript. Let me state as a caveat to any further comments here that none of these dynamics is right or wrong; as David Morgan stressed in our classes on focus groups last summer, focus groups are robust. The groups discussed the same sorts of topics both when the group was heavily reliant on the moderator, and when the group carried on its own conversation with little moderator input, and even against moderator requests.

There were patterns of conversation in some groups which were not consistent with the purposes and advantages of focus group methods: in a focus group, all members should participate, and there should be a common discussion in which the participants respond not only to the moderator but also to each other. In discussing this, I will mention the names of particular groups, as examples--this does not mean that these were the only groups where these particular dynamic patterns emerged. Problems can be seen when one pattern of conversation dynamics dominated an entire transcript. The clearest example of this is the transcript for Ivankiv Ukrainian men, where most exchanges are m/p1/p2/p1/p2/p1/p2. This means that the discussion was primarily an argument between two participants, and in fact three participants in that group said almost nothing. The two participants who dominated this group had a very interesting discussion that raised many issues related to the economy, nationality, and environment, but the moderator did not intervene in their dialogue or create opportunities for others to speak. Many of the groups followed a fairly consistent m/p/m/p/p pattern, in which the moderator raised questions and solicited individual answers. For example, in Bukhara Uzbek women, the moderator established this pattern very firmly at the beginning; and participants continued to expect that the moderator would call on them to speak. In this group, the moderator’s dominant position in directing questions and sequences meant that participants often did not respond directly to the comments of others. There were several focus groups in which the dominant pattern throughout the transcript was m/p/p/p/p or p/p/p/p. This pattern, found in Tallinn Estonian women, indicated that the moderator was not very consistent in directing the conversation or in making "probe" questions; instead, participants took the conversation in their own directions, with the moderator occasionally intervening. In the ideal focus group, a balance needs to be struck between moderator-directed discourse and participant freedom to comment and exchange opinions.

The brief section at the beginning of each transcript in which participants introduce themselves can serve as point of comparison between cases when the moderator prompts the participants to mention certain kinds of information, and cases when participants volunteer that information without prompting--or, alternatively, maintain silence. An examination of the occurrence of mention of family status in personal introductions will suggest the influence of a moderator’s prompting on the kinds of information that participants give. The most consistent approach to introductions was taken in all of the Uzbekistan focus groups, where moderators prompted participants to introduce themselves by talking about their work and their families. Moderators either asked for this information directly, or modeled it by introducing themselves with mention of work and family. Participants all introduced themselves with mention of their work and family status, following the moderator’s prompt.

In the Estonia focus groups, the approach to introductions was inconsistent. In one transcript (Tamsallu Estonian men) there is no evidence of any introductions at all. In a number of the groups, participants were simply asked to say a few words about themselves. In two groups (Tallinn Estonian Women and Tartu Estonian women) the moderator prompted participants to mention family, once by directly asking, and once by mentioning her marriage and daughter in her own introduction. In five groups (Narva Russian men and women, Sillimae Estonian men and women, Tartu Estonian men), moderators did not ask about families in introductions, but some of the participants mentioned their families. After one participant had mentioned family in his/her introduction, all of the following participants also mentioned family. In four groups (Tallinn Russian men, Tallinn Russian women, Tallinn Estonian men, Tamsallu Estonian women) moderators did not prompt for mention of family status in asking participants to introduce themselves, and the participants made no mention of families. These latter two clusters indicate that a moderator prompt is not a feature which absolutely controls group, but rather, influences them, and a similar influence can come from participants’ voluntary speeches. That is, in nine groups the moderator did not prompt for families, but families were mentioned anyway by participants in five of those groups. The introduction of family by one participant was enough to prompt all remaining participants to also talk about their family status. Another example of the influence of a prompt in the introduction section involves the mention of citizenship. In some of the Russian Estonia groups, though not all of them, moderators prompted participants to include in their introduction mention of where they came from. The Russian groups in Estonia which were so instructed then highlighted citizenship issues throughout their discussion, while those Russian groups which were not so instructed were also not so quick to raise citizenship issues.

To round out this picture of consistency and inconsistency in introductions, the Ukraine focus groups, like the Estonia groups, asked for participant introductions with a variety of prompts. I won’t mention every group here, but in Lviv Ukrainian men and Donetsk Russian men, the moderator did not prompt for family, and the participants introduced themselves only with reference to work and education. In Kyiv Ukrainian men, Lviv Ukrainian women and Ivankiv Ukrainian men, the moderator did not ask about families, but as soon as one participant mentioned his or her family in the introduction, all of the following participants did so as well. In Ivankiv Ukrainian women, Kyiv Ukrainian women and Donetsk Ukrainian women, moderators prompted for mention of families, and participants then mentioned their families in their introductions.

If one were to take the introductions from these focus groups and use them to suggest that women talk more about families than men do, this would be a thesis that cannot be proven. All participants talked about families when they were prompted to do so. When there were not prompted to talk about families, some women’s groups and some men’s groups talked about families anyway. When the moderator did not prompt for mention of family status, the volunteering of that information by one participant, whether male or female, was sufficient to prompt mention of family in the remaining introductions in his or her group.

Likewise, there was significant moderator impact on other topics raised in personal introductions. In some groups, moderators simply asked for introductions, and allowed participants to say one sentence each. In other groups, moderators asked follow-up questions to elicit more information. In Ivankiv Ukrainian men, the moderator followed almost each brief introduction with some additional question, concerning age, education, or work history. This gave one participant the grounds to describe his complex life history, from place of birth through recent discharge from the army. Likewise, follow-up questions in some of the Estonia Russian groups elicited stories about when participants came to Estonia, and immediately, their attitudes and concerns about citizenship. In the Uzbekistan groups none of these sorts of narratives were expressed, because the moderators specifically stated the kinds of information that they wanted in introductions, and rarely asked any follow-up questions.

Moderators on all sites were asked to use the same set of questions, and the same kinds of follow-up probes. However, in different sites, the questions and probes were handled in differing orders, and this had some bearing on the structure of the conversation, and on the amount of information that the moderator elicited. In asking about improvements, moderators were to ask what sort of improvements participants had noticed in the past ten years. Moderators were to follow this with some examination of those improvements, with questions like: has this been an improvement for everyone, or has it only affected some people? why have these changes taken place?

There was inconsistency in the way that the moderator phrased the question concerning improvements, and this had significance in shaping the kinds of improvements that participants did and did not mention. In Lviv Ukrainian men, the moderator asked in what ways political and economic changes of the last ten years improved the lives of participants and people like them. In a clarification, he stressed that these were changes that arose from socio-political changes. In a further clarification, he told participants that the changes could be on a personal level or a state level. In this group, members consistently mentioned changes on the level of society such as open borders, not having to serve in Soviet army, freedom of expression and of religion; they did not talk about intimate concerns such as marriage or the birth of a child. By contrast, in the Kyiv Ukrainian men group, the moderator asked about changes in the country which had affected the participants’ private lives in a positive way. When participants asked for clarification, the moderator stressed that these were improvements which "are tied more to yourselves, although they are related to social life". When asked to talk about their improvements, most of the Kyiv Ukrainian men mentioned events like the birth of a child, and obtaining a new apartment; these comprised about half of the improvements they mentioned, while the other half were improvements on a larger social scale, like independence, freedom of thought, peace and so forth. I chose these examples--both involving Ukrainian men-- specifically to demonstrate that the moderator’s framing of the basic question on improvements had a very significant influence over the mention of personal improvements versus social improvements. Participants in more women’s groups than men’s groups were prompted by moderators in this particular question to talk about personal improvements. In one group, Ivankiv Ukrainian women, the women only talked about personal improvements. In Tashkent Uzbek women and Ferghana Uzbek women, the moderator gave equal weight to personal and social improvements in her introduction of this topic, and the women then mentioned as many personal improvements as social improvements. In light of the fact that moderators framed this question differently in different groups, it would be difficult to prove any assertions that discourses about private and family life should be expected more in women’s groups than in men’s focus groups. When the moderator emphasized improvements in society in women’s focus groups, women were more likely than men to talk about improvements that affected children, in general, than men were. But discourses about private and family life were produced both in men’s and women’s groups when moderators specifically elicited them by stressing the personal when introducing the topic of improvements.

The structure of questioning when eliciting improvements also varied from group to group, and from country to country, and this may have had a significant influence over the relative prominence which topics received. Again, the moderators in Uzbekistan focus groups were the most consistent in their approach to eliciting participants’ mention of improvements. Moderators in these groups told participants to mark their most important improvement. They turned to the first participant, asked for his or her most important improvement, wrote it down, and then turned to the second participant, and so on. The only follow-up questions were for clarification of expression. When all of the "most important" improvements had been written, the moderator proceeded around the group, one by one, until he or she accumulated a list of all of the improvements. Rarely did the moderator ask the question "does anyone have something similar. " While participants were encouraged to mention items that were the same as those mentioned by others, the moderators put higher value on making sure that every participant had his or her turn to speak and to mention every improvement that he or she listed. In many cases, this resulted in long and varied lists of improvements. [In Uzbekistan, Uzbek and Tajik groups accumulated lists of between 8 and 18 improvements, while the Russian groups listed 6 improvements each, Karakalpak men named 5 improvements and Karakalpak women named 4 improvements.]

In the second level of questioning, consistency in the Uzbekistan groups broke down. In most groups, but not all, participants were asked to rank the improvements, voting for the one they considered most important. In many groups, moderators asked participants to explain why improvements came about, and who benefited. However, in some groups (for example Bukhara Tajik and Uzbek women), participants were asked instead to explain how they understood a particular improvement, or what affect it had on them personally. In Tashkent Uzbek women, the moderator asked each participant in what way she had personally experienced this improvement, rather than approaching the question "who benefited" in a more general way. In a number of the Uzbekistan groups, the discussion of improvements really went no further than listing and describing improvements. The moderator was in control of the structure of questioning; and in these groups the very brief mention of many topic, rather than a fuller elucidation or an open, contentious discussion, is more attributable to the moderator's influence--the follow-up probes that moderators did NOT make--than to the participants’ interest or lack thereof in the topics they raised.

In the Ukraine and Estonia focus groups, the approach to eliciting improvements was far less consistent and more eclectic. In Ivankiv Ukrainian men, the model followed was closest to the model used in the Uzbekistan focus groups: moderators asked participants to mark the most important improvement, and then proceeded around the circle one by one, eliciting improvements and assembling a list. The moderator made no follow-up probes at all, until one participant mentioned that one could now be proud to be Ukrainian, at which the moderator asked whether others concurred, sparking a discussion. In Kyiv Ukrainian men, the moderator also went around the circle assembling a list, but in this case, the moderator had not asked participants to speak briefly, or to mention their most important improvement first. In the only case of its kind, the Kyiv Ukrainian men each introduced all of their improvements in a long speech which included a full defense of the reasons that this improvement was important, and some allusion to its causes. After these speeches, the moderator made no follow-up probes at all. In Narva Russian women, the moderator did not ask women to mark their most important improvement, and assembled her list by asking one woman for an improvement, then asking if others had something similar, and then going on to the next improvement. The moderator of this group did ask follow up questions: who benefited? (but not, why did these changes take place)

In all other groups in Ukraine and Estonia, the procedure for eliciting improvements gave more emphasis to follow-up questions, and far less to assembling a full list of improvements or ranking those improvements. For example, in Lviv Ukrainian women, the moderator first elicited two improvements and asked others for similar topics. She then asked participants: why has this improved your life? After some discussion, she turned to another participant for mention of another improvement. She asked other follow-up questions after other improvements, such as "why is this important?" Participants mentioned five improvements, and the moderator had them vote on the most important. In the discussion of which was most important, the participants themselves turned the conversation to the reasons that the improvements came about, relieving the moderator of responsibility for using this as a probe question.

In most of the Estonia groups, the most consistent element in section eliciting improvements was the moderator’s use of the follow-up question, "is this improvement available to everyone?" In most of the Estonia groups, the moderator followed one of two patterns. In one model he/she elicited one improvement and asked if others had something similar and then opened discussion of this particular improvement with the follow up "does this affect everyone" before eliciting any more improvements. In the second model he/she elicited a cluster of several improvements, and then opened discussion of that group of improvements before going on to elicit more improvements for the list. Both of these patterns were followed in Ukraine as well. The extreme example of this approach can be seen in the Tartu Estonian men discussion. The first participant mentioned as an improvement that now one did not have to know the Russian language, because Estonian is the official language. The moderator followed up with a question that emphasized the problems of this change: "the rights of Estonians have increased and the rights of Russians have decreased, so you could say that this positive change does not affect everyone in Estonia." This launched a long discussion of language issues and the Estonian men’s views of Russian attitudes toward learning Estonian and becoming citizens, a topic which recurred throughout the transcript again and again. One wonders whether participants would have deemed this such a prominent issue if they had first been asked to assemble a list of their improvements and rank which one was most important. Perhaps this issue would still have occupied the majority of this focus group, but the structure of questioning certainly had some affect the amount of time spent on this topic. After this lengthy discussion, the moderator again elicited another improvement, and asked such follow-up questions as "don’t you see any problems here?," and "why has it happened?" Eventually participants named seven improvements; the moderator allowed open discussion of all, but did not ask his own follow-up questions concerning causes or who benefited about all of the topics. In this, he had good company; in almost no group was there a consistent pattern of follow-up questioning concerning improvements.

There is no single ideal structure of questioning; each structure had its advantages and disadvantages. The pattern used in the Uzbekistan groups led to the assembling of long lists of improvements, far longer than common in the other countries. This is certainly not because the groups in Uzbekistan have lives that are so much more wonderful than elsewhere; it was because the moderators were very thorough in eliciting and listing all improvements that participants had written down. However, in assembling and ranking improvements before (or in some cases, instead of) discussing and following up on improvements as they were mentioned, the Uzbekistan groups produced sections on improvements which contain many brief mentions of many topics, but very few extended discourses about any particular improvements. On the other hand, groups in Ukraine and Estonia that followed the other method of eliciting improvements--elicit one, discuss it thoroughly, and then go on--assembled less complete lists of participants improvements, with some participants even being reluctant to mention theirs. However, these groups discussed particular improvements at greater length, and with greater variety in follow-up questioning. The length of discussion of any particular issue may not have correlated with its relative importance in participants’ minds, as a group, but with their importance to one person in the group, and with the kinds of follow-up questions that the moderator chose to make, or not to make.

With all of these systematic difference, at what level is there commonality and comparability in the Improvements section of the transcripts? With some caveats, most groups did bring up and discuss improvements that affect their society. Mention of some topics as "improvements" is fairly common across all sites, and differences in emphasis from country to country and nationality to nationality have some bearing for interpreting identity formation.

Most moderators--and participants--seemed to regard the discussion of problems to be more important than the discussion of improvements. Within comments on improvements in focus groups from all countries, participants repeatedly turned the conversation toward a discussion of problems. Moderators often cut short discussions of improvements, asking making few or no follow-up probes, and sometimes pointing out that their turn toward problems reflected the participants’ obvious interest.

The framing of the "problems" section was less problematic than that of the "improvements" section, in part because participants had already clarified the parameters of the question--personal or social--in the "improvements" section, and did not need to have them repeated. In a few of the groups, the question was framed very simply: name three problems or deteriorations in your life and the lives of people like you in the past 10 years. Some moderators added definitions of these changes. One of these framings provoked a minor rebellion; in Lviv Ukrainian men, the moderator asked participants to name political and economic changes that have caused difficulties. The participants in the group, who began the whole discussion with the very common assertion that there many not have been any improvements in the last ten years, now refused to find any problems. Political changes had not caused any difficulties, they asserted; there were ecological problems, perhaps, but not political problems. The moderator then reframed the question to ask, "Is there anything in your lives that has become more complicated?" At this, participants managed to think of three problems, and one or two more came up in the course of their discussion.