Reading and analysing textfiles and building sting arrays

Question

Hello community I'm working on a project were I need to read several types of measurement files (basically text). In most cases the files consist of a header and several additional lines like this: Orginator blah blah Descriptor blah blah ........... ........... begin_data 0 1 2 3 4 5 6 7 8 9 ....... X end_data What I want is to: 1. Open the file 2. Analyze the file and search for strings named begin_data and end_data 3. Remove everything except the numbers between end_data and begin_data 4. Built a string array constisting only of the numbers then split the string using   as separator The begin_data and end_data strings are allways present but may vary in position. Gratefull for any ideas and suggestions. Vladimir

Nitin Sontakke · Answer

Hope this helps! public Double[] parseFile(String filename) { List returnValue = new List (); String inputLine = String.Empty; Boolean inData = false ; Double doubleValue; Boolean success; const String MARKER_BEGIN = BEGIN_DATA ; const String MARKER_END = END_DATA ; if (!File.Exists(filename)) { //File not found return empty array. return returnValue.ToArray(); } using (StreamReader sr = new StreamReader(filename)) { while (!sr.EndOfStream) { inputLine = sr.ReadLine(); if (!inData) { if (inputLine == MARKER_BEGIN) { inData = true ; continue ; } } if (inData) { if (inputLine == MARKER_END) { break ; } String[] values = inputLine.Split( new Char[] { ' '  '	' }); foreach (String value in values) { success = Double.TryParse(value out doubleValue); if (success) { returnValue.Add(doubleValue); } } } } sr.Close(); } return returnValue.ToArray(); }

Vladimir Gajic · Answer

Hello Nitin after analysing and implementing your code properly it works real fine. I get exactly what i need to get. Even if the first character after BEGIN_DATA starts with a word (e.g. color name) it works fine since everything that is not a nmber is obviously ignored. Thank you for your valuable time and efforts. Regards Vladimir.

Vladimir Gajic · Answer

Hello Nitin you are right. Maybe I explained my task badly. So sorry! Cheers Vladimir.

Nitin Sontakke · Answer

Please correct me if I am wrong here. Your original file had a tab as a separator. Why would need so much code then? Can we not just return the row as is? All you have to do is appropriately identify the row after we have found the BEGIN_DATA tag. Am i right?

Vladimir Gajic · Answer

Hello Nitin the separator between the numbers in the original file file is allways a single space charakter or a tab. There is a very strict specification for generating such files. Cheers Vladimir.

Nitin Sontakke · Answer

It's very simple. I honestly do not mean to confused you but at the same think that you should be given an option. There are two ways to go about it: 1/ I hope your separator is fixed for example a comma or a pipe charactor. Tell me that and I will modify the code accordingly. Even you can do that. Instead of building the List of doubles keep concatanating the string. (This is recommended option from my side!) 2/ If your separator is fixed you can always use Join() function to convert array to string with your choice of separator. (Not so recommended) I will leave it to you. Ping should you need any help.

Vladimir Gajic · Answer

Hello Nitin the file I provided is correct. In some cases however the line after BEGIN_DATA starts with a sample name (e.g. clor name) followed by the numbers. This I can remove easily. My current problem is that the result of your code is a single row array without separators between the numbers. What I need is that the numbers be arranged in a single collumn. That's what I'm trying to figure-out how to split the values without a separator. or am I completely wrong? Regards Vladimir.

Nitin Sontakke · Answer

I had tested the code with the file provided. If it is not working for some reason please provide the file. I already know where it will fail and if you tell me to handle the scenarios I will but there has to be a pattern of course. I encourage you to do that of course! It will fail the line with numbers is NOT right after BEGIN_DATA tag. I had assumed that it will be the first line after BEGIN_DATA. Your sample had it that way. You may need to implement a logic to ignore all such rows and identify a line with numbers. The doubles SHOULD HAVE some separator. Otherwise how would we know?

Vladimir Gajic · Answer

Hello again Nitin made a mistake while implementing your code... stupid of me. It works well except that the returned double[] is a single row array with no separators between the numeric values. The original file has separated values. Codeline line 33 is declaring a separator but it don't seems to work. I try to figure out how to solve this. Cheers Vladimir.

Vladimir Gajic · Answer

Hello Nitin I finished analysing your code. The method returns definetly a string with zero-members. The split-option also works wrong because the result is a single line string without separators between values. Could it be that returnValue is valid only within the Streamreader codeblock? Thanks again

Vladimir Gajic · Answer

Hello Nitin I've tried to implement your code. It returns a array of 0 lenght and all values are set to zero too. Trying to figure-out what went wrong.. Basically the code seems ok to me. Appreciate your help. Vladimir.

Vladimir Gajic · Answer

Hello Nitin thanks a lot for your efforts! The code seems good to me so far. I'll try to implement this evening and I will report. Vladimir.

Nitin Sontakke · Answer

Is just something as follows is fine by you? If yes I will go ahead. public double[] getValues(string filename) { }

Vladimir Gajic · Answer

Hello Amit the result of the file analyses will be a double[] array for further computation of various colorimetric values (the input values are spectral reflection values of a colour at certain wavelenghts). The input values don't need to be stored or depicted in any way. I shall use them later in my code to draw a typical spectral plot. I just need the routine to read the files propper. As a matter of fact using the common Linq functions etc. makes it possible to read a certain file-structure where my numeric values start at say row 19. The exception will occure when the values are in a row different then 19. The primary file structure is allways the same the header may vary depending on the file orginator (varios solutions for driving spectrophotometers). Thanks anyway for your interest. I'm open to any suggestions.

Amit Gupta · Answer

@ Vladimir According to my point of view loading your text files in a richtextbox control will makes your task easier it also provides 'find' features so that data can be fetched easily. Although I had made similar several types of application in banking sector. You had not mentioned the output of the files (where it should be depicted). I suggest you to create a temp DataTable and keep all the required output in it and then there will be several possibilities to export in xml or excel etc. Do let us know if we can assist you further in this.

Vladimir Gajic · Answer

Hello Nitin thanks for your effort! Attached is a sample file. The begin_data and end_data markers appear only once in every file. The numeric values are allways in one line.

Nitin Sontakke · Answer

I can give you a code to begin with. However I would have been really nicer to have a sample file with dummy data. The reason is simple. You didn't mention if it is only single set of begin data + end data or multiple. If multiple should they be clubed together or kept separate. If kept separate whould would be the key and how to fetch it?