Alignment based on item structure

class alignment.countryspecificrequest.ESSCountrySpecificR01

Item names of country specific questions in ESS round 1.

class alignment.countryspecificrequest.ESSCountrySpecificR02

Item names of country specific questions in ESS round 2.

class alignment.countryspecificrequest.ESSCountrySpecificR03

Item names of country specific questions in ESS round 3.

class alignment.countryspecificrequest.ESSCountrySpecificR04

Item names of country specific questions in ESS round 4.

class alignment.countryspecificrequest.ESSCountrySpecificR05

Item names of country specific questions in ESS round 5.

class alignment.countryspecificrequest.ESSCountrySpecificR06

Item names of country specific questions in ESS round 6.

class alignment.countryspecificrequest.EVSCountrySpecificR03

Item names of country specific questions in EVS round 3.

class alignment.countryspecificrequest.EVSCountrySpecificR04

Item names of country specific questions in EVS round 4.

alignment.alignment_based_on_item_structure.align_introduction_instruction_request(df, df_source, df_target, item_type)

Aligns introduction, instruction and requests segments. Differently from response segments, these other item types can’t be merged. There are five distinct cases to consider: 1) only source segments (df_target is empty), 2) only target segments (df_source is empty), 3) df_source has more segments than df_target 4) df_target has more segments than df_source and, 5) df_source and df_target have the same number of segments.

Parameters

• df (param1) – dataframe to store the questionnaire alignment.

• df_source (param2) – dataframe containing the data of the source questionnaire (always English).

• df_target (param3) – dataframe containing the data of the target questionnaire

• item_type (param4) – metadata that indicates if the dataframes contain introductions, instructions or requests.

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.align_more_segments_in_source(list_source, list_target, sorted_aligments, source_segments_without_pair, df)

Calls the appropriate method for alignment with more segments in source dataframe, concerning the number of pairless segments. This method is called from a broader sets of cases contained in prepare_alignment_with_more_segments_in_source. If there is only one pairless source segment, call treat_a_single_pairless_source(), otherwise call treat_multiple_pairless_source_segments()

Parameters

• list_source (param1) – list of source segments (contains segments of same item_name and item_type)

• list_target (param2) – list of target segments (contains segments of same item_name and item_type)

• sorted_aligments (param3) – sorted list segments aligned via best match strategy.

• source_segments_without_pair (param4) – indexes of pairless source segments

• df (param5) – dataframe to store the questionnaire alignment

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.align_more_segments_in_target(list_source, list_target, sorted_aligments, target_segments_without_pair, df)

Calls the appropriate method for alignment with more segments in target dataframe, concerning the number of pairless segments. This method is called from a broader sets of cases contained in prepare_alignment_with_more_segments_in_source. If there is only one pairless target segment, call treat_a_single_pairless_target(), otherwise call treat_multiple_pairless_target_segments()

Parameters

• list_source (param1) – list of source segments (contains segments of same item_name and item_type)

• list_target (param2) – list of target segments (contains segments of same item_name and item_type)

• sorted_aligments (param3) – sorted list segments aligned via best match strategy.

• target_segments_without_pair (param4) – indexes of pairless target segments

• df (param5) – dataframe to store the questionnaire alignment

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.align_on_metadata(df, df_source, df_target, process_responses)

Calls the appropriate method for alignment based on metadata. Responses are aligned separately of other item types because answers are merged using the item value. :param param1 df: dataframe to store the questionnaire alignment :type param1 df: pandas dataframe :param param2 df_source: dataframe containing the data of the source questionnaire (always English). :type param2 df_source: pandas dataframe :param param3 df_target: dataframe containing the data of the target questionnaire :type param3 df_target: pandas dataframe :param param4 process_responses: indicates if the response segments should be processed, country-specific answers are excluded by design. :type param4 process_responses: boolean

Returns

df (pandas dataframe) with newly aligned survey items.

alignment.alignment_based_on_item_structure.align_responses(df, df_source, df_target)

Aligns response segments by merging them on item_value metadata. :param param1 df: dataframe to store the questionnaire alignment :type param1 df: pandas dataframe :param param2 df_source: dataframe containing the data of the source questionnaire (always English). :type param2 df_source: pandas dataframe :param param3 df_target: dataframe containing the data of the target questionnaire :type param3 df_target: pandas dataframe

Returns

df (pandas dataframe) with newly aligned response segments.

alignment.alignment_based_on_item_structure.different_source_target_index_card_instructions(source_index, target_index, aux_source, aux_target, list_source, list_target, item_type, df)

Align source and target instruction segments identified as showcard segments. For sake of simplicity, when the source index and the target index of showcard segments are different, they are moved to the first position in the isntruction list.

Parameters

• source_index (param1) – source showcard segment.

• target_index (param2) – target showcard segment.

• aux_source (param3) – auxiliary list of source segments being modified in outer loop (contains segments of same item_name and item_type).

• aux_target (param4) – auxiliary list of target segments being modified in outer loop (contains segments of same item_name and item_type).

• list_source (param5) – list of source segments (contains segments of same item_name and item_type).

• list_target (param6) – list of target segments (contains segments of same item_name and item_type).

• item_type (param7) – item_type metadata, can be REQUEST, INTRODUCTION or INSTRUCTION

• df (param8) – dataframe to store the questionnaire alignment

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.filter_by_module(df_source, df_target, module)

Filters the source and target dataframes by the module that is being currently analyzed.

Parameters

• df_source (param1) – dataframe containing the data of the source questionnaire (always English).

• df_target (param2) – dataframe containing the data of the target questionnaire

• module (param3) – questionnaire module being currently analyzed in outer loop.

Returns

df_source (pandas dataframe) and param2 df_target (pandas dataframe). Source and target dataframes filtered by the module specified by parameter.

alignment.alignment_based_on_item_structure.find_best_match(list_source, list_target, item_type)

Finds the best match for source and target segments (same item_type) based on the lenght of the segments. :param param1 list_source: list of source segments (contains segments of same item_name and item_type) :type param1 list_source: list :param param2 list_target: list of target segments (contains segments of same item_name and item_type) :type param2 list_target: list

Returns

alignment (list). Alignment pair represented by the index of target and source segments being aligned (index 0 = target,index 1 = source), selected with lenght of the segments strategy.

alignment.alignment_based_on_item_structure.get_original_index(list_source, list_target, source_segment_index, target_segment_index, aux_source, aux_target)

Gets the original index of aligned segments, as the auxiliary lists are being modified and the indexes does not correspond to the original ones. :param param1 list_source: list of source segments (contains segments of same item_name and item_type). :type param1 list_source: list :param param2 list_target: list of target segments (contains segments of same item_name and item_type). :type param2 list_target: list :param param3 source_segment_index: source segment aligned in aux_source list. :type param3 source_segment_index: int :param param4 target_segment_index: target segment aligned in aux_target list. :type param4 target_segment_index: int :param param5 aux_source: auxiliary list of source segments being modified in outer loop (contains segments of same item_name and item_type). :type param5 aux_source: list :param param6 aux_target: auxiliary list of target segments being modified in outer loop (contains segments of same item_name and item_type). :type param6 aux_target: list

Returns

original_index_target (int), original_index_source (int). Original indexes (in list_source, list_target) of source/target segments aligned.

alignment.alignment_based_on_item_structure.get_study_metadata(filename)

Get study metadata embedded in filename. It can be retrieved either from the source or the target file.

Parameters

filename (param1) – name of either source or target file.

Returns

study (string). Metadata that identifies the study of the questionnaires that are being aligned.

alignment.alignment_based_on_item_structure.get_target_language_country_metadata(filename)

Get target language/country metadata embedded in filename, to name the output aligned file.

Parameters

filename (param1) – name of target file.

Returns

target_language_country (string). Metadata that identifies the language/country of the target questionnaire being aligned.

alignment.alignment_based_on_item_structure.identify_showc_segment(list_source, list_target, item_type)

Searches in list_source, list_target if there are intructions that seem to be show card segments that should be aligned together. This method was implemented as an additional strategy to align correctly instruction segments. :param param1 list_source: list of source segments (contains segments of same item_name and item_type) :type param1 list_source: list :param param2 list_target: list of target segments (contains segments of same item_name and item_type) :type param2 list_target: list :param param3 item_type: item_type metadata being analyzed. In this method we want to consider only instruction segments. :type param3 item_type: string

Returns

str indicating if instructions that follow the show card segments were found.

alignment.alignment_based_on_item_structure.instantiate_country_specific_request_object(study)

Instantiates the appropriate set of country-specific requests according to the study. Country-specific requests are deleted from alignment by design because the answer categories frequently change from country to country.

Parameters

study (param1) – study metadata, embedded in filenames.

Returns

country_specific_requests (Python object). Instance of python object that encapsulates the item names of the country specific questions.

alignment.alignment_based_on_item_structure.only_one_segment_in_source_align(alignment, source_segment, target_segment, list_target, aux_target, df)

Fills the dataframe with remaining target segments that do not have source correspondencies. This method is called when the dataframe contains one source segment to two or more target segments. The alignment pair is defined in the find_best_match() method and the remaining target segments are included in this method, respecting the structure order. :param param1 alignment: Alignment pair represented by the index of target and source segments being aligned :type param1 alignment: list :param (index 0 = target: :param index 1 = source): :param selected with lenght of the segments strategy.: :param param2 source_segment: source segment that has a match according to find_best_match(). :type param2 source_segment: list :param param3 target_segment: target segment that has a match according to find_best_match(). :type param3 target_segment: list :param param4 list_target: list of target segments (contains segments of same item_name and item_type) :type param4 list_target: list :param param5 aux_target: list of source segments, excluding the target_segment (contains segments of same item_name and item_type) :type param5 aux_target: list :param param6 df: dataframe to store the questionnaire alignment :type param6 df: pandas dataframe

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.only_one_segment_in_target_align(alignment, source_segment, target_segment, list_source, aux_source, df)

Fills the dataframe with remaining source segments that do not have target correspondencies. This method is called when the dataframe contains one target segment to two or more source segments. The alignment pair is defined in the find_best_match() method and the remaining source segments are included in this method, respecting the structure order. :param param1 alignment: Alignment pair represented by the index of target and source segments being aligned :type param1 alignment: list :param (index 0 = target: :param index 1 = source): :param selected with lenght of the segments strategy.: :param param2 source_segment: source segment that has a match according to find_best_match(). :type param2 source_segment: list :param param3 target_segment: target segment that has a match according to find_best_match(). :type param3 target_segment: list :param param4 list_source: list of source segments (contains segments of same item_name and item_type) :type param4 list_source: list :param param5 aux_source: list of source segments, excluding the source_segment (contains segments of same item_name and item_type) :type param5 aux_source: list :param param6 df: dataframe to store the questionnaire alignment :type param6 df: pandas dataframe

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.prepare_alignment_with_more_segments_in_source(df, list_source, list_target, item_type)

Calls the appropriate method for alignment with more segments in source dataframe, concerning the number of pairless segments

Parameters

• df (param1) – dataframe to store the questionnaire alignment

• list_source (param2) – list of source segments (contains segments of same item_name and item_type)

• list_target (param3) – list of target segments (contains segments of same item_name and item_type)

• item_type (param4) – item_type metadata, can be REQUEST, INTRODUCTION or INSTRUCTION

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.prepare_alignment_with_more_segments_in_target(df, list_source, list_target, item_type)

Calls the appropriate method for alignment with more segments in target dataframe, concerning the number of pairless segments

Parameters

• df (param1) – dataframe to store the questionnaire alignment

• list_source (param2) – list of source segments (contains segments of same item_name and item_type)

• list_target (param3) – list of target segments (contains segments of same item_name and item_type)

• item_type (param4) – item_type metadata, can be REQUEST, INTRODUCTION or INSTRUCTION

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.same_source_target_index_card_instructions(source_index, target_index, aux_source, aux_target, list_source, list_target, item_type, df)

Align source and target instruction segments identified as showcard segments.

Parameters

• source_index (param1) – source showcard segment.

• target_index (param2) – target showcard segment.

• aux_source (param3) – auxiliary list of source segments being modified in outer loop (contains segments of same item_name and item_type).

• aux_target (param4) – auxiliary list of target segments being modified in outer loop (contains segments of same item_name and item_type).

• list_source (param5) – list of source segments (contains segments of same item_name and item_type).

• list_target (param6) – list of target segments (contains segments of same item_name and item_type).

• item_type (param7) – item_type metadata, can be REQUEST, INTRODUCTION or INSTRUCTION

• df (param8) – dataframe to store the questionnaire alignment

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.treat_a_single_pairless_source(list_source, list_target, sorted_aligments, source_segments_without_pair, df)

Align source and target segments, in case where there is only one source segment without a pair. :param param1 list_source: list of source segments (contains segments of same item_name and item_type) :type param1 list_source: list :param param2 list_target: list of target segments (contains segments of same item_name and item_type) :type param2 list_target: list :param param3 sorted_aligments: sorted list segments aligned via best match strategy. :type param3 sorted_aligments: list :param param4 source_segments_without_pair: index of pairless souce segment :type param4 source_segments_without_pair: int :param param5 df: dataframe to store the questionnaire alignment :type param5 df: pandas dataframe

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.treat_a_single_pairless_target(list_source, list_target, sorted_aligments, target_segments_without_pair, df)

Align source and target segments, in case where there is only one target segment without a pair. :param param1 list_source: list of source segments (contains segments of same item_name and item_type) :type param1 list_source: list :param param2 list_target: list of target segments (contains segments of same item_name and item_type) :type param2 list_target: list :param param3 sorted_aligments: sorted list segments aligned via best match strategy. :type param3 sorted_aligments: list :param param4 target_segments_without_pair: index of pairless target segment :type param4 target_segments_without_pair: int :param param5 df: dataframe to store the questionnaire alignment :type param5 df: pandas dataframe

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.treat_multiple_pairless_source_segments(list_source, list_target, sorted_aligments, source_segments_without_pair, df)

Align source and target segments, in case where there are multiple source segments without a pair. :param param1 list_source: list of source segments (contains segments of same item_name and item_type) :type param1 list_source: list :param param2 list_target: list of target segments (contains segments of same item_name and item_type) :type param2 list_target: list :param param3 sorted_aligments: sorted list segments aligned via best match strategy. :type param3 sorted_aligments: list :param param4 source_segments_without_pair: indexes of pairless souce segments :type param4 source_segments_without_pair: list :param param5 df: dataframe to store the questionnaire alignment :type param5 df: pandas dataframe

Returns

df (pandas dataframe) with newly aligned survey item segments.

alignment.alignment_based_on_item_structure.treat_multiple_pairless_target_segments(list_source, list_target, sorted_aligments, target_segments_without_pair, df)

Align source and target segments, in case where there are multiple target segments without a pair. :param param1 list_source: list of source segments (contains segments of same item_name and item_type) :type param1 list_source: list :param param2 list_target: list of target segments (contains segments of same item_name and item_type) :type param2 list_target: list :param param3 sorted_aligments: sorted list segments aligned via best match strategy. :type param3 sorted_aligments: list :param param4 target_segments_without_pair: indexes of pairless target segments :type param4 target_segments_without_pair: list :param param5 df: dataframe to store the questionnaire alignment :type param5 df: pandas dataframe

Returns

df (pandas dataframe) with newly aligned survey item segments.