Django: add an object to a model from json values dynamically

The latest thing I had to add to the pipeline management system is the ability to record dynamic information in different models depending on the software used. For example, I need to store the filename used as input for the ls command. Each software is called from a Celery task,  and I want that information is recorded only when the software has been properly executed. Assuming that I have the following models:

class Software(models.Model):
    name = models.CharField(max_length=200)
    executable = models.CharField(max_length=200) 
    #json {parameter_name:'parameter type'}
    parameters = models.TextField(null=True, blank=True) 
    #json {event_name:{event_type:[{type_name:{field_1:'value 1'}}]}}
    events = models.TextField(null=True, blank=True)

    def build_commandline(self, parameters_value):
        commandline = self.executable
        parameters = json.loads(self.parameters)
        for parameter_name in parameters.keys():
            if parameter_name in parameters_value.keys():
                if parameters[parameter_name]['type'] == 'argument':
                    commandline += ' ' + parameters_value[parameter_name]
                # other types, for example a switch like '-a',
                # or options like '-n count' in the 'head' command
        return commandline


class UsedFile(models.Model):
    path = models.CharField(max_length=255)
    file_type = models.CharField(max_length=20, null=True, blank=True)
    description = models.CharField(max_length=255)

and the instance “ls” as:

ls = Software(name="ls",
              executable="ls",
              parameters=json.dumps({'path':{
                                         'type':'argument',}, 
                                     })
ls.save()                        

I want to add the ‘path’ parameter value to the UsedFile model in the ‘path’ field.
The first thing to do is to add reserve a name for the event type, for example ‘on_end’, and to add a trigger after the software is executed:

import os
import shlex

'''
Execute a software instance with a lot of parameters
'''
def execute_software(software, parameters={}):
    cline = software.build_commandline(parameters)
    # result: 'ls /home/caio_giulio/'
    try:
        output = subprocess.check_call(
            shlex.split(str(cline)))
    except subprocess.CalledProcessError as cpe:
        # something wrong
        print "oh noes!"
        print "Process failed: exit status %s" % cpe.returncode
        return False
    software_events = json.loads(software.events)
    if 'on_end' in software_events.keys():
        manage_actions(software_events['on_end'], parameters)
    return True

software = Software.objects.get(name='ls')
# retrieve the directory from somewhere, es. '/home/caio_giulio/'
path = request.POST.get('directory', os.getcwd())
# build the command line dynamically from the parameters 
parameters = {'path':path}
execute_software(software, parameters)

Now finally I can add the event actions to the ‘ls’ software object. Supposing that in the future I will add other features, I reserved the keyword ‘models’ for the actions that involve models. The first thing to do is to classify the parameters value in two different classes, one that retrieve the value from the software parameters and the other that add a custom value (for example a description of the used software).
The shortest way is to prepend a simple tag to the values, for example ‘parameter:’ and ‘custom:’.
The ‘ls’ software on_end field value then becomes:

file_model = {'UsedFile':{
                          '_app_name': <name of the application that includes UsedFile>,
                          'path':'parameter:path',
                          'file_type':'',
                          'description':'custom:Used by: ls'}}
models = [file_model]
on_end['models'] = models
ls.on_end = on_end
ls.save()

The last thing to do is to implement the ‘manage_actions’ procedure.

from django.db.models import get_model
def manage_actions(actions, software_parameters):
    if 'models' in actions.keys():
        # for each model in list
        for action_model in actions['models']:
            for model_name in action_model.keys():
                # e.g. get_model('SoftwareApplication', 'UsedFile')
                dynamic_model = get_model(action_model[model_name]['_app_name'],
                                          model_name)
                # remove the reserved key for the application, to allow a for loop in all
                # the parameters 
                del action_model[model_name]['_app_name']
                values = {}
                for field_name in action_model[model_name].keys()
                    field_value = ''
                    field_dict = action_model[model_name][field_name].split(':', 1)
                    # e.g. ['custom','Used by: ls'], ['parameter','path']
                    if field_dict[0] == 'custom':
                        field_value = field_dict[1]
                    # double check to avoid errors
                    if field_dict[0] == 'parameter' and field_dict[1] in parameters.keys():
                        field_value = parameters[field_dict[1]]
                    if field_value != '':
                        values[field_name] = field_value
                # example: values = {'path': '/home/caio_giulio',
                #                    'description': 'Used by: ls'}
                dynamic_value = dynamic_model.objects.create(**values)
                '''
                like the hard-coded 
                used_file = UsedFile(path = '/home/caio_giulio',
                                     description = 'Used by: ls')
                used_file.save()
                '''
                

Hey yo!
Now every time that we execute ‘ls’ the ls input path is automatically added to the UsedFile table.
Homework: create a set of templates and views that allow to add different models to the events json field.

Pubblicità