C# 2Gb file is 4gb in Ram. Why? -
im reading in file(this file consists of 1 long string 2gb in length).
this function read contents of file memory , splits string , places: *reader = streamreader
public list<char[]> getallcontentaslist() { int bytestoread = 1000000; char[] buffer = new char[bytestoread]; list<char[]> results = new list<char[]>(); while (_reader.read(buffer, 0, bytestoread) != 0) { char[] temp = new char[bytestoread]; array.copy(buffer,temp,bytestoread); results.add(temp); } return results; }
when data in placed list takes 4gb in ram. how possible when file 2gb in size?
*edit
this ended doing. im not converting array of bytes string, im passing bytes on manipulating them. fiel 2gb in mem instead of 4gb
public list<byte[]> getallcontentaslist() { int bytestoread = 1000000; var buffer = new byte[bytestoread]; list<byte[]> results = new list<byte[]>(); while (_reader.read(buffer, 0, bytestoread) != 0) { //string temp = encoding.utf8.getstring(buffer); byte[] b = new byte[bytestoread]; array.copy(buffer,b,bytestoread); results.add(b); } return results; }
educated guess here:
the file utf-8 or ascii encoded , (mostly) contains singly byte wide characters (or possibly other codepage single byte wide).
now, .net characters utf-16 2 (or more) bytes in length.
so, in memory characters double size.
Comments
Post a Comment