0

Disclaimer

First off, this is one of my first questions on SE, so I apologize if I break any written or unwritten conventions.

Introduction

At my job in a kind of customer service/support role, I often have to check the log files of one our REST interfaces (JSON format).

This log contains the requests made on one of our webservices. The data is statistically interesting to us, so at some point one of our developers made a tool which extracted data from the log and returned a CSV file which can be used for data manipulation.

However, one of our other customers (on another server) uses the web service a bit differently which causes the tool to not be able to process the data. The thing is that our developers are very busy and this is not a high-priority issue. Still, it strikes me as if this should be fairly simply, so I would like to make a good impression and maybe build this myself.

Example

Example input (JSON):

2017-01-31 05:42:43:587 -> 2017-01-31 05:42:43:587:1536:GET /v1/webservice_path.../ABC123 X-Api-Key:xyz  
2017-01-31 05:42:43:587 <- 2017-01-31 05:42:43:587:1536:OK:{
    "parameter 1": true,
    "parameter 2": true

Example output (CSV):

ID;Parameter 1;Parameter 2
ABC123;true;true

I have technical flair but absolutely no formal training in coding apart from the most basic HTML, Python and Ruby courses on CodeAcademy.

Question

This is actually a two-part question:

  1. How/where do I learn to make such a tool/script? Which language, which channel to learn from etc.
  2. Is it even realistic for me to try to be able to build this? And is the time consumed learning this worth it?

Extra info

I want to build the tool myself; not copy in a solution one of you might be able to make in minutes. One reason for this is that the use of our web service varies from customer to customer, so another customer might be using it like this (which I also want to be prepared for):

2017-01-31 05:42:43:587 -> 2017-01-31 05:42:43:587:1536:GET /v1/webservice_path.../ ID_1:ABC123 ID2_:QWE456 X-Api-Key:xyz  
2017-01-31 05:42:43:587 <- 2017-01-31 05:42:43:587:1536:OK:{
    "parameter 1": true,
    "parameter 2": true
    "parameter 3": true

(please note that instead of placing request directly after /, the request is entered in field "ID_1" and an optional extra field "ID_2" also contains data)

  • Technically as the closing } is missing neither is valid JSON. Having space separated URLs looks a bit strange as for building it you would just to decide what language you want to use and look into the corresponding documentation/tutorials. As you said you did some basic python you could look into Parsing values from a JSON file using Python?. – Seth Jan 31 '17 at 11:51
  • The closing { is in the file, but I missed it when I c/p'ed it. Thank you for your comment, will definitely look into that question! – Grobsrop Jan 31 '17 at 12:09
  • I've been able to successfully extract and parse JSON formatted data with Python before using with open(RawDataFile,"r") as file: data = json.load(file) and then with open(CsvFile,"w",newline='') as file: csv_file = csv.writer(file) for a simple and easy solution. I didn't really have much Python experience until I started working on that and even tried Powershell and other solutions. The simplest solution I found in my case was Python so I'd look into that for sure. If you can provide a better example from a link or a dump, I may be able to help more with the syntax you'd use. – Pimp Juice IT Jan 31 '17 at 13:22
  • 1
    Issues specific to programming and software development are off topic, see On-Topic. Try Stack Overflow but please first read How do I ask a good question?. – DavidPostill Jan 31 '17 at 15:00
  • Thank you for the feedback David. Having read the guideline you linked to I am not entirely sure what needs to be changed in order for this to be considered a good question. Can you help me in this? Also, what is preferred: that I copy the question and ask it on SO or that the question is migrated? – Grobsrop Jan 31 '17 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.