C# Binaryreader Position

Question

Hi All Bit stuck.. Is there a way to read in a certain length of bytes search for a specific string and then set the binaryreader's location to that point? For example.. Read in 40000 bytes find hello set basestream.position at that location Many thanks

Vulpes · Answer

I've changed the first version of the program to look for SAM etc and it's working fine: using System; using System.IO; using System.Text; using System.Text.RegularExpressions; class Program { static void Main() { // create a file containing 20000 'A's followed by SAM etc and 20000 'B's string filePath = @c:\users\jay\jay.bin; // say byte[] sam = Encoding.ASCII.GetBytes(SAM\0South American\0\0World); using (BinaryWriter bw = new BinaryWriter(File.Open(filePath FileMode.Create) Encoding.ASCII)) { byte[] ba1 = new byte[20000]; for (int i = 0; i bw.Write(ba1 0 20000); bw.Write(sam 0 sam.Length); byte[] ba2 = new byte[20000]; for (int i = 0; i bw.Write(ba2 0 20000); } // now read back first 40000 bytes and look for SAM etc using (BinaryReader br = new BinaryReader(File.Open(filePath FileMode.Open) Encoding.ASCII)) { byte[] ba3 = br.ReadBytes(40000); // now look for SAM etc string regExp = @SAM.South American.+?World; Match m = Regex.Match(Encoding.ASCII.GetString(ba3) regExp); if (m.Success) { br.BaseStream.Position = m.Index; // set file position to that location Console.WriteLine(Found SAM etc at position {0} m.Index); Console.Write(As expected the next 9 bytes (in hex) are: ); for (int i = 0; i Console.WriteLine(); } else { Console.WriteLine(SAM etc was not found); } } Console.ReadKey(); } } The output is: Found SAM etc at position 20000 As expected the next 9 bytes (in hex) are: 53 41 4D 00 53 6F 75 74 68

Jay S · Answer

Finally managed to get this to work with your solution!

Jay S · Answer

I don;t think it's going to work. What you've said makes sense and should work I just think there's something fundamentally wrong with either i) what i'm doing or ii) my file

Jay S · Answer

ok..so it's matching (again) but i'm still getting the next byte as 0 and the byte after that as 77 which do not relate to the word SA (first 2 bytes from SAM)

Vulpes · Answer

You'll need either: @SAM.South American.+?World or @SAM\x00South American.+?World The first one will be happy if there's any character between SAM and South. The second one will be happy if the only character between those words is a null character (ASCII 0).

Jay S · Answer

So would i just need: @SAM South American +?World

Vulpes · Answer

If it's a real dot it would be 2E in hex. As it's 00 it must be a null byte which your hex editor will show as a dot. If that's the case the regular expression will not match which might explain why you're getting 'funny' results.

Jay S · Answer

I'm not sure if there is a dot between SAM and South. In hex SAM.South = 53 41 4D 00 6F 75 74 68 Where the 00 is the dot?

Vulpes · Answer

Most hex editors show bytes outside the 'printable' range 32 to 126 as just dots. As it happens the '.' is the Regex symbol which matches any character except new line (
). To match: SAM.South American...............World where there could be any number of characters (though at least one) between 'American' and 'World' would require the regular expression: string regExp = @SAM\.South American.+?World Notice that the dot after SAM needs escaping by preceding it with a backslash because it's a real dot not a special symbol.

Jay S · Answer

Something isn't right... I do: byte[] ba3 = binaryReader.ReadBytes(100000); Match m = Regex.Match(Encoding.ASCII.GetString(ba3) SAM.South American); if (m.Success) { binaryReader.BaseStream.Position = m.Index; // set file position to that location byte tester = binaryReader.ReadByte(); } So if the match index sets it at the start of what its found i'd expect tester = S or 41 in Hex but doesn't it returns 0 and the next byte returns 77 not 4D(hex)

Jay S · Answer

If I'm looking for the string: SAM.South American (the  .  is shown in the ACSII part of my hex editor dont know if i need it?) will it try and do an exact match?

Vulpes · Answer

Here's the program in my first post updated to now use File.ReadAllBytes rather than BinaryReader. The output is of course the same as before. using System; using System.IO; using System.Text; using System.Text.RegularExpressions; class Program { static void Main() { // create a file containing 20000 'A's followed by 'hello' and 20000 'B's //string filePath = @c:\users\jay\jay.bin; // say string filePath = @jay.bin; // say byte[] hello = Encoding.ASCII.GetBytes(hello); using (BinaryWriter bw = new BinaryWriter(File.Open(filePath FileMode.Create) Encoding.ASCII)) { byte[] ba1 = new byte[20000]; for (int i = 0; i bw.Write(ba1 0 20000); bw.Write(hello 0 5); byte[] ba2 = new byte[20000]; for (int i = 0; i bw.Write(ba2 0 20000); } // now read whole file back and look for hello in first 40000 bytes byte[] ba3 = File.ReadAllBytes(filePath); Match m = Regex.Match(Encoding.ASCII.GetString(ba3 0 40000) hello); int index = -1; if (m.Success) { index = m.Index; // set array position to that location Console.WriteLine(Found 'hello' at position {0} m.Index); Console.Write(As expected the next 5 bytes are: ); for (int i = 0; i Console.WriteLine(); // do something with rest of bytes from index + 5 onwards in array } else { Console.WriteLine('hello' was not found); } Console.ReadKey(); } }

Jay S · Answer

Can i embed all these reads? e.g: I read in the whole file using binary.reader -- do some stuff -- Use what we have discussed to then read in x bytes (e.g to EOF) Do the regex match -- do some stuff Reading x bytes to end of file again -- do some stuff etc etc?

Vulpes · Answer

Yes 'count' is the integer parameter which BinaryReader.ReadBytes takes. So if you pass 40000 it will read the next 40000 bytes or if less the rest of the bytes in the file. You would of course only use File.ReadAllBytes if you knew from the start that you wanted to read all the bytes in the file into memory. However notice that it's easy to search only a part of the resulting byte array using regular expressions because Encoding.GetString has an overload which allows you to specify a start index and a length: http://msdn.microsoft.com/en-us/library/05cts4c3(v=vs.100).aspx

Jay S · Answer

So replace count with a number so is it just count? Don't think I can use System.IO.File.ReadAllBytes(filePath) as I've already read in my whole file and with what we've discussed its manipulating part way through a file..

Vulpes · Answer

That's right. On the second point BinaryReader.ReadBytes(count) will just read to the end of the file if there are less than 'count' bytes remaining. However if you just want to read the entire file the easiest way is to forget BinaryReader and use instead: System.IO.File.ReadAllBytes(filePath) which does what it says on the tin :)

Jay S · Answer

I see...so regardless if i am matching a string or a hex string as its all ASCII its all the same. Instead of reading in for example 40000 bytes is there a way to read to the EOF?

Vulpes · Answer

An ASCII hex character is represented in regular expressions using the syntax '\xnn' where 'nn' represents the two hex digits. The Regex.Match (string regexp) method starts at the beginning of the string and looks for the first occurrence of the regular expression. If it finds one the Index property of the Match object returns the index of the start of the match (zero based of course). If you're interested in all matches it's possible to get them by calling the Match object's NextMatch() method successively or (better) to use the Regex.Matches method which gets them all at once. Here's a quick example which illustrates these points: using System; using System.Text.RegularExpressions; class Test { static void Main() { string text = abracadabra; string regExp = @\x61\x62; // ab MatchCollection mc = Regex.Matches(text regExp); foreach(Match m in mc) { Console.WriteLine({0} found at index {1} m.Value m.Index); } Console.ReadKey(); } } The output is: ab found at index 0 ab found at index 7

Jay S · Answer

....It would still be possible to represent those hex bytes in string form and search for them using a regular expression.... How would i do this? You regex match works very nicely without any impact on performance. What I tried to search for was SAM.South American...............World (As this is what it appeared in my hex editor). Though it matched i'm unsure if i sets the index at the start of the string or the end of the string/

Vulpes · Answer

Here's the same program but using a custom method to look for the bytes rather than a regular expression. Performance still seems OK: using System; using System.IO; using System.Text; class Program { static void Main() { // create a file containing 20000 'A's followed by 'hello' and 20000 'B's string filePath = @c:\users\jay\jay.bin; // say byte[] hello = Encoding.ASCII.GetBytes(hello); using (BinaryWriter bw = new BinaryWriter(File.Open(filePath FileMode.Create) Encoding.ASCII)) { byte[] ba1 = new byte[20000]; for (int i = 0; i bw.Write(ba1 0 20000); bw.Write(hello 0 5); byte[] ba2 = new byte[20000]; for (int i = 0; i bw.Write(ba2 0 20000); } // now read back 40000 bytes look for 'hello' and set file position to beginning of that string using (BinaryReader br = new BinaryReader(File.Open(filePath FileMode.Open) Encoding.ASCII)) { byte[] ba3 = br.ReadBytes(40000); // now look for hello long index = FindBytes(ba3 hello); if(index > -1) { br.BaseStream.Position = index; Console.WriteLine(Found 'hello' at position {0} index); Console.Write(As expected the next 5 bytes are: ); for (int i = 0; i Console.WriteLine(); } else { Console.WriteLine('hello' was not found); } } Console.ReadKey(); } static long FindBytes(byte[] source byte[] search) { if (search.Length > source.Length) return -1; for (long i = 0; i { bool found = true; for (long j = i; j { if (search[j - i] != source[j]) { found = false; break; } } if (found) return i; } return -1; } }

Vulpes · Answer

Using regular expressions will (roughly) treble memory usage because you have to convert the ASCII byte stream to a unicode string (2 bytes per character) before you can do the search. It would still be possible to represent those hex bytes in string form and search for them using a regular expression. If the memory usage hit is too high you'd need to write a custom method to search the byte array for whatever consecutive bytes you're looking for. To my knowledge there's no built-in method in the .NET framework which can do this. This shouldn't be too difficult but would probably be slower than using regular expressions.

Jay S · Answer

That's an interesting method reading the whole file in as a string.. Whats the implications on memory usage to do this? What if instead of looking for the string hello I want to search for a hex value(s) eg 53 41 4D 00 53 6F 75 74 68 20 41 6D 65 72 69 63 61 6E 00 00 00 00 00 00 00 00 00 00 00 00 00 06 00 57 6F 72 6C 64

Vulpes · Answer

Try this: using System; using System.IO; using System.Text; using System.Text.RegularExpressions; class Program { static void Main() { // create a file containing 20000 'A's followed by 'hello' and 20000 'B's string filePath = @c:\users\jay\jay.bin; // say byte[] hello = Encoding.ASCII.GetBytes(hello); using (BinaryWriter bw = new BinaryWriter(File.Open(filePath FileMode.Create) Encoding.ASCII)) { byte[] ba1 = new byte[20000]; for (int i = 0; i bw.Write(ba1 0 20000); bw.Write(hello 0 5); byte[] ba2 = new byte[20000]; for (int i = 0; i bw.Write(ba2 0 20000); } // now read back first 40000 bytes and look for 'hello' using (BinaryReader br = new BinaryReader(File.Open(filePath FileMode.Open) Encoding.ASCII)) { byte[] ba3 = br.ReadBytes(40000); // now look for hello Match m = Regex.Match(Encoding.ASCII.GetString(ba3) hello); if (m.Success) { br.BaseStream.Position = m.Index; // set file position to that location Console.WriteLine(Found 'hello' at position {0} m.Index); Console.Write(As expected the next 5 bytes are: ); for (int i = 0; i Console.WriteLine(); } else { Console.WriteLine('hello' was not found); } } Console.ReadKey(); } } The output should be: Found 'hello' at position 20000 As expected the next 5 bytes are: 104 101 108 108 111