Building a data mart to analyse web log traffic
I recently had the opportunity to build a data mart in order to analyse web logs from IIS servers. The solution utilised SQL Server 2012 Database Engine, Integration Services, Analysis Services and Excel 2010 (to slice/dice the data). Loading IIS logs (text files with fixed width spaces) was fairly challenging due as the column definitions would alter throughout. I therefore used the .NET StreamReader class to read the metadata and detect changes in the fields.
Read more...