How to use unittest module for a python script which calls mutiple functions and uses global variables - python-unittest

Question

This is one of the function in my python script for which I am trying to write unit test case, since it uses global variables and audit and big query functions which is written as different utility scripts I am not understanding how to write #patch and execute unit test cases for the same.
How will I patch global variables?
How to patch functions which doesn't have any return for eg :audit_event_source_table, can we ignore such functions during unit testing ? if so how to do the same?
How to do assertion as I do not have any return value but have logger.info messages?
import logging
from datetime import datetime
from pathlib import Path
import sys
import __main__
from intient_research_rdm_common.utils.audit_utils import audit_event_source_table, audit_event_job_table, \
get_job_id, get_source_object_id
from intient_research_rdm_kg_core.common_utils.utils.bigquery_utils import bigquery_data_read
from intient_research_rdm_kg_core.common_utils.utils.conf_read import read_args, read_source_config, read_env_config
global project_id, service_account, conn_ip, debug, node_table_list, edge_table_list, source_name
def edge_validation():
global edge_table_list
global source_name
edge_table_na = []
edge_table_list_rowcount_zero = []
dataset_e = "prep_e_" + source_name
row_count = 0
edge_table = ""
source_object_start_timestamp = datetime.now()
source_object_id = get_source_object_id(source_name, source_object_start_timestamp)
source_object_type = AUDIT_SOURCE_OBJECT_TYPE_BIGQUERY
job_id = get_job_id(source_object_start_timestamp)
source_object_name = dataset_e
try:
for edge_table in edge_table_list:
sql_query = " SELECT * FROM " + "`" + project_id + "." + dataset_e + ".__TABLES__` WHERE table_id =" + "'" + edge_table + "'"
data_read, col_names = bigquery_data_read(service_account, sql_query, project_id)
for ind in data_read.index:
row_count = (data_read['row_count'][ind])
if len(data_read.index) == 0:
edge_table_na.append(edge_table)
elif row_count == 0:
edge_table_list_rowcount_zero.append(edge_table)
if len(edge_table_na) > 0:
logging.info("Missing Edge tables in preprocessing layer {} ".format(edge_table_na))
if len(edge_table_list_rowcount_zero) > 0:
logging.info("Edge tables with row count as zero in Pre-processing layer {} ".format(edge_table_list_rowcount_zero))
if len(edge_table_na) == 0 and len(edge_table_list_rowcount_zero) == 0:
logging.info(
"Edge list validation for the source {} has been successfully completed with no discrepancies".format(
source_name))
audit_event_source_table(source_object_id, source_object_name, source_object_type, source_name,
source_object_name,
job_id, AUDIT_JOB_STATUS_PASS, source_object_start_timestamp,
datetime.now(), 'NA', 'NA', project_id)
if len(edge_table_na) > 0 or len(edge_table_list_rowcount_zero) > 0:
audit_event_source_table(source_object_id, source_object_name, source_object_type, source_name,
source_object_name,
job_id, AUDIT_JOB_STATUS_PASS, source_object_start_timestamp,
datetime.now(), 'NA', 'NA', project_id)
sys.exit(1)
except Exception as e:
msg = '{} : Issue with the edge validation for {} is: \n{}\n'.format(datetime.now(), edge_table, e)
logging.error(msg)
audit_event_source_table(source_object_id, source_object_name, source_object_type, source_name,
source_object_name,
job_id, AUDIT_JOB_STATUS_FAIL, source_object_start_timestamp,
datetime.now(), AUDIT_ERROR_TYPE_PREPROCESSING_KG_LAYER_VALIDATION, msg,
project_id)
raise Exception(msg)

Answer 1

Patch global variables - in the same way that you patch a method of a class, you patch the global variable. It's not clear in your code snippet where the global variables are defined (ie. do you import these variables from another module or do you assign to those variables at the top of your Python script). Either way, you patch in the namespace where the function is being used. If you can confirm further details I will be able to assist.
Personally, the way I patch and test functions with no return value is the same. For example, if I wanted to patch the source_object_start_timestamp variable, I would use: source_object_start_timestamp = patch('pandas.datetime.utcnow', return_value="2020-08-16 20:36:06.578174").start(). For BigQuery functions, I would still patch them but in your unit test, use the mock_call_count method of the unittest.mock.mock class to test if that function has been called.
Point 2 addresses your third query - use the mock_call_count method to check how many times the mock has been called