Commit 5404382a authored by sparonuz's avatar sparonuz
Browse files

Squashed commit of the following:

commit 5ae2ecd02e265fa20160621ab6357fb795841623
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Fri Feb 14 16:25:43 2020 +0100

    Merged branch

commit 086e8a98
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Wed Feb 5 10:50:15 2020 +0100

    Now openMP clause are kept and not removed as comments.

commit 75905614
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Mon Feb 3 11:13:20 2020 +0100

    Really was lucking a new line at the en of file.

commit e7dba4e0
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Thu Jan 30 17:20:50 2020 +0100

    Comment at the beginning of the Namelist were causing errors.

commit 3b3b13a3
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Wed Jan 29 18:20:41 2020 +0100

    BUGFIX: regex was not working in case of a declaration of a REAL without kind specification.

commit c603ab9a
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Wed Jan 29 17:13:29 2020 +0100

    Fix for declaration of parameters in Fortran 77 style.

commit dbec6923
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Wed Jan 29 11:49:26 2020 +0100

    Moved some definition out of the function, where they belongs

commit 5be26f2e
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Wed Jan 29 11:45:26 2020 +0100

    Workaround for store variable of a routine which contains subroutine declaration

commit 680f5943
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Tue Jan 28 18:57:49 2020 +0100

    Fixed function 'find_next_end' in case of nested declaration.

commit 8c7aafb2
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Tue Jan 28 16:07:01 2020 +0100

    Fix for function which real retval was specified on the same line of the declaration.

commit ec427b82
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Tue Jan 28 10:14:02 2020 +0100

    Fixed regex for counting do loops end.

commit 622b7734
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Mon Jan 27 16:07:44 2020 +0100

    In case of array assignation pass split with '=' to avoid confusion with parenthesis when dimension of array are specified ( type(rpe_var), parameter :: var(1:3) = ... )

commit b160dd5f
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Mon Jan 27 13:20:52 2020 +0100

    BugFIX: changes in  commit a4db3364 were not saved in line obj.

commit a4db3364
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Mon Jan 27 12:55:57 2020 +0100

    RPE variable  used for declaring other RPE var are now correctly parsed.

commit 7ae0652c
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Mon Jan 27 12:31:20 2020 +0100

    Added function to check if a declaration has intrinsics, and the corresponding fix.

commit f2598946
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Fri Jan 24 18:16:47 2020 +0100

    Implemented function to detect instrinsics: still the names are hardocoded.

commit 73961a9b
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Fri Jan 24 13:13:39 2020 +0100

    BUGFIX: no operation is possible at declaration time btw rpe variables. Coversion is done after the operation is applied to the operands.

commit 92d0a28b
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Fri Jan 24 09:39:14 2020 +0100

    Fixed declaration of variables which contains operations, like: zibv0 = one/0.74_jprb.

commit 2b7fa969
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Thu Jan 23 13:03:53 2020 +0100

    Fixed declaration of arrays Fortran 2003 style, square brackets.

commit 5bc3874e
Author: Oriol Tintó-Prims <oriol.tinto@bsc.es>
Date:   Wed Jan 22 16:17:28 2020 +0100

    Small fix to the function that loads the list of files to keep unmodified.

commit 9ba01854
Author: Oriol Tintó-Prims <oriol.tinto@bsc.es>
Date:   Wed Jan 22 14:52:13 2020 +0100

    Fix in parameter reshape exception.

commit f090e419
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Tue Jan 21 17:24:22 2020 +0100

    parse_attributes now returns also implicitly declared dimensions.

commit b67fff85
Author: sparonuz <stella.paronuzzi@bsc.es>
Date:   Tue Jan 21 10:48:17 2020 +0100

    exlcuding the "contiguous::" assertion from the attributes of object variable "routines".
parent eb3deb01
# Import classes and functions from RPE_Utils
from UtilsRPE.MainFunctions import parse_sources, postprocess_sources, assign_sbits_to_rpe_variables, \
create_read_precisions_module_and_add_to_file, replace_wrkalloc, preprocess_sources
create_read_precisions_module_and_add_to_file, preprocess_sources
from UtilsRPE.Functions import load_vault, load_list_of_files_to_keep_unmodified
if __name__ == "__main__":
......@@ -9,20 +9,15 @@ if __name__ == "__main__":
path_to_preprocessed_sources = "./ModelSources/Processed/"
path_to_output_sources = "./ModelSources/Output/"
# Starting
print("Starting the implementation of the reduced precision emulator in:\n %s\n\n" % path_to_input_sources)
# !!!!!!Only for NEMO!!!!!!! Probably should be removed from the main tool.
# Remove the use of wrkalloc calls and fix it with allocates
replace_wrkalloc(path_to_input_sources, path_to_preprocessed_sources)
# Blacklist of files that should not be processed. Is read from file "list_of_files_to_keep_unmodified.txt"
# List of files that should not be processed, as red from "list_of_files_to_keep_unmodified.txt"
preprocess_blacklist = load_list_of_files_to_keep_unmodified()
# The code is first processed to replace real declarations with type(rpe_var)
preprocess_sources(path_to_preprocessed_sources, path_to_preprocessed_sources, blacklist=preprocess_blacklist)
# Format text + replace real declarations with type(rpe_var)
preprocess_sources(path_to_input_sources, path_to_preprocessed_sources, blacklist=preprocess_blacklist)
# The code is parsed to obtain source information
# Obtain source information
vault = parse_sources(path_to_preprocessed_sources, save_vault=True)
# We load the code information into the vault
......
This diff is collapsed.
This diff is collapsed.
......@@ -7,14 +7,13 @@ from .Classes import Interface, Vault, Module, DerivedType, ObtainSourcefileInfo
from .Functions import insert_variable_precision_specification_to_allocatable, \
insert_variable_precision_specification_to_namelist_parameter, insert_variable_precision_to_routine_variable, \
add_after_call_routines, load_source_files, listdir_ord, split_vars_by_allocatable, split_vars_by_main_or_routine, \
split_by_appearance_in_namelist, replace_workalloc_in_file, preprocess_file, postprocess_file
split_by_appearance_in_namelist, postprocess_file
def parse_sources(path_to_model_sources, save_vault=False):
# Find the nemo source-files in sources
# (Using the sources that already modified replacing real declarations by type(rpe_var) )
sources = [f for f in listdir_ord(path_to_model_sources) if
isfile(join(path_to_model_sources, f)) and f.count(".f90")]
isfile(join(path_to_model_sources, f)) and f.count(".f90")]
# Initialize two empty dictionaries that will contain all the routines and all the functions
object_oriented_dictionary = {"procedures": {}, "interfaces": {}, "variables": {}, "modules": {},
"derivedtypes": {}}
......@@ -88,26 +87,28 @@ def parse_sources(path_to_model_sources, save_vault=False):
return vault
def replace_wrkalloc(path_to_input_sources, path_to_output_sources):
# Find files in sources
all_sources = [f for f in listdir_ord(path_to_input_sources) if
isfile(join(path_to_input_sources, f)) and f.count(".f90")]
# Process Sources!
total = len(all_sources)
for index, filename in enumerate(all_sources):
print("\rReplacing wrkallocs %i/%i %20s" % (index+1, total, filename), end="")
replace_workalloc_in_file(path_to_input_sources, path_to_output_sources, filename)
def preprocess_sources(path_to_input_sources, path_to_output_sources, blacklist=[]):
# Find files in sources
from .Classes import ImplementRPE
# read all files sources excluding hidden files ( like vim swp files)
all_sources = [f for f in listdir_ord(path_to_input_sources) if
isfile(join(path_to_input_sources, f)) and f.count(".f90") and f not in blacklist]
# Process Sources!
for index, filename in enumerate(all_sources):
total = len(all_sources)
print("\rReplacing reals %i/%i %20s" % (index+1, total, filename), end="")
preprocess_file(path_to_input_sources, path_to_output_sources, filename)
isfile(join(path_to_input_sources, f)) and f.count(".f90") and f[0][0] != "."]
if len(all_sources):
# Load and format files
for index, filename in enumerate(all_sources):
f = ImplementRPE(join(path_to_input_sources, filename), None)
total = len(all_sources)
# Remove comments, empty lines and old style declarations
if filename in blacklist:
print("\rFormatting file %i/%i %20s" % (index + 1, total, filename), end="")
f.pre_process()
# Additionally reals are replaced with rpe_var type
else:
print("\rFormatting file and replacing reals %i/%i %20s" % (index + 1, total, filename), end="")
f.pre_process(replace_reals=True)
f.save_modified_file(join(path_to_output_sources, filename))
else:
# Sources is an empty folder
raise ValueError("No files found in %s folder", path_to_input_sources)
def postprocess_sources(path_to_input_sources, path_to_output_sources, vault):
......@@ -208,4 +209,4 @@ def create_read_precisions_module_and_add_to_file(path_to_sources, target_module
with open(join(path_to_sources, target_module), "w") as output_file:
output_file.write(original_source)
else:
warnings.warn("File %s not found in sources.\nUnable to insert read_precisions module")
\ No newline at end of file
warnings.warn("File %s not found in sources.\nUnable to insert read_precisions module")
......@@ -3,18 +3,9 @@ import re
# Function only used in the case of NEMO to replace the wrkalloc calls with actual allocations.
def replace_wrkalloc(text):
def remove_comments(_text):
# Function to remove the comments
_lines = _text.split("\n")
for _line in _lines:
comments = _line.find("!")
if comments > -1:
new_line = _line[:comments]
_text = _text.replace(_line, new_line)
return _text
# Find if the argument is a variable or a dimension
def argument_type(_argument, _text):
# Find if the argument is a variable or a dimension
try:
int(_argument)
except ValueError:
......@@ -33,8 +24,8 @@ def replace_wrkalloc(text):
return "u",
# Get the dimensions and the arguments of the wrk_alloc call"
def strip_arguments(_line, _text):
# Get the dimensions and the arguments of the wrk_alloc call"
comments = _line.find("!")
if comments > -1:
_line = _line[:comments]
......@@ -44,7 +35,6 @@ def replace_wrkalloc(text):
arguments = [x.strip() for x in new_line.split(",")]
dimensions = []
variables = []
_text = remove_comments(_text)
for argument in arguments:
type = argument_type(argument, _text)
......@@ -70,6 +60,7 @@ def replace_wrkalloc(text):
list_of_things_to_replace = [["dimension", "DIMENSION"], ["pointer", "POINTER"], ["wrk_alloc (", "wrk_alloc("],
["wrk_dealloc (", "wrk_dealloc("]]
text = text.replace("USE wrk_nemo", "")
match = re.search('\n[^\!]*call *wrk_alloc *\(', text, flags=re.I)
if match:
# Replace some strings defined above to simplify the process
......@@ -149,8 +140,5 @@ def replace_wrkalloc(text):
new_line += dealloc + "\n"
text = text.replace(lines[n], new_line)
# Fix substituting pointers inside TYPE definition
# Divide the text by lines
lines = text.split("\n")
return text
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment