 |
 | Home | Contact Us | FAQ | Search & Site Map | Link to Us |
 | Sign In | Join | Other 45 Sites in Network |
SQL Server Forum / General / Data Warehousing / May 2006Is a Data Warehouse guy a "report writer"?
A while back, I had the worst 6 month period of my life. It was awful, every day I would come home from work in a bad mood and stressed. Every day I would go to work dreading yet another day. Why did my life suck so bad? Because I was a full time query writer. Every day I would get requests for ad hoc queries to be written and would be resonsible for getting them out. I hated those requests, I hated the stress, I hated the whole "this report is the most inportant thing in the whole world" mentality that all of the people making these requests had. It was awful. I went back to being just a plain ol' DBA, and absolutely love it. I really, really love it. I now need to add some new skills to my arsenal. I want to either get into Data Warehousing, or learn another DB platform in addition to MSSQL. Based on my background and what I do and don't like, could anyone make any recommmendations between the two? Will I encounter the same scenarios as a DW guy that I did as a report writer? TIA, ChrisR. I dont think any job title could explicitly save you from something like this. You should have more then enough of work regarding DW, but if theres noone else to do reports and you know how to do them....... MC >A while back, I had the worst 6 month period of my life. It was awful, >every [quoted text clipped - 18 lines] > > TIA, ChrisR. Solution: Give the end-users a reporting tool... That in itself can be a fun project :)  SignatureAdam Machanic Pro SQL Server 2005, available now http://www.apress.com/book/bookDisplay.html?bID=457 -- >I dont think any job title could explicitly save you from something like >this. You should have more then enough of work regarding DW, but if theres [quoted text clipped - 26 lines] >> >> TIA, ChrisR. Or forget how to do reports ;). I did a similar thing a couple of years back. Wanted to be a DBA so I forgot how to use .NET. It was a hard going for a while, but now I know my way around SQL Server, BI, a little about DM and so on since all the time I work with data. And I still dont know how to use .NET if any1 asks ;). MC > Solution: Give the end-users a reporting tool... That in itself can be a > fun project :) [quoted text clipped - 29 lines] >>> >>> TIA, ChrisR. Hi Chris, well, I am a BI person and have been for 15 years.....I can appreciate the whole 'this report is the most important report in the world and I want it done now' thing.....years ago I was one of the advocates of developing tools to give end users the ability to write their own reports, and we lost the arguement.....hence your unfortunate position..... If DBA type people want to move forward in the BI world but do not want to do the end user reporting bit there is always an interesting role in designing BI data models and designing ETL subsystems......unless you want to become an Oracle DBA... ;-) In my experience BI Data models and ETL subsystems are frequently built very poorly and there is something of a career in fixing them...however, these things are 'plumbing' and they are not seen to be as 'sexy' and some wonderful report......though most of the world seems to be completely unaware that if the data is not available it cannot be put onto the report......funny thing that!! If you are interested and have the inclination I also suggest to people to get closer to the business problems actually being solved by BI systems....the fact is that more and more IT jobs will be shipped off to places like India/China and IT folks in western (high salary) countries need to find work higher up the food chain. Solving the real business problems like driving new revenues and increasing profitability are some of the business problems that can be significantly assisted by intelligently using information. Just producing more reports rarely has a positive impact on the bottom line....;-) Peter Nolan www.peternolan.com I must be the odd duck. I'm the new db admin-slash-report guru guy for a major corporation and I'm loving it. Every day is a challenge, some new wall to run into repeatedly, and I'm enjoying the heck out of it. Stressful? Yep. demanding? You betcha. The $$$ ain't bad either. Makes it all worthwhile. ; ) Randall Arnold > Hi Chris, > well, I am a BI person and have been for 15 years.....I can appreciate [quoted text clipped - 29 lines] > Peter Nolan > www.peternolan.com > If DBA type people want to move forward in the BI world but do not want > to do the end user reporting bit there is always an interesting role in > designing BI data models and designing ETL subsystems......unless you > want to become an Oracle DBA... ;-) Is there such a thing as a designer that doesnt have to get too involved in the reporting end of it? SignatureTIA, ChrisR > Hi Chris, > well, I am a BI person and have been for 15 years.....I can appreciate [quoted text clipped - 29 lines] > Peter Nolan > www.peternolan.com Hi Chris, absolutely...that is what I have done for years.......nowadays I am back into report development to an extent.....but there is absolutely a role in a BI project for a DW Architect where the architect must know and understand how the various reporting tools work and what their limitations are, and design the database to support those limitations......yes, we are still designing databases around the shortcomings of our ETL tools and BI tools...I wrote a newsletter on that once....(www.peternolan.com)... Personally, it is my opinion that the job of designing and performing the ETL subsystems and data models makes a MUCH bigger difference in the possible future value that can be derived from the ODS/EDW. However, this is architecturue and infrastructure.....it is plumbing....and no-one cares about the plumbing until it does not work...and then no query tool can recover the loss of confidence in the data. I have spent a good part of the last 14 years developing faster ways or building the back ends..... We can now do 5-10 more than we could do in the same time 10 years ago. My latest effort to is make it possible to generate 95%+ of all executable objects required for the ETL subsystem from the mapping spreadsheet....by being able to generate the ETL subsystem entirely from the mapping spreadsheet we can guarantee the consistency of the ETL and the documentation from the mapping spreadsheet....we also put the mapping spreadsheet into SQL Server and published a suite of reports with report services......the suite put together is now, by far, the fastest and most effective way to build the most difficult piece of an EDW....the ETL subsystem.......what we have done is a revolution that will change the way the world builds ETL subsystems.......and we are working on the next piece of the revolution now... :-) Peter Thanks so much for your insights. I've been doing MS Analysis Services tutorials and really do have an interest in BI as it's quite a new challenge, but did not want to get sucked back in to being a report writer. Thanks for the clarification. > Hi Chris, > absolutely...that is what I have done for years.......nowadays I am [quoted text clipped - 33 lines] > > Peter Sounds like you'll love the new reporting services in SQL Server 2005. Then the business can create adhoc reports to their heart's content. > Thanks so much for your insights. I've been doing MS Analysis Services > tutorials and really do have an interest in BI as it's quite a new [quoted text clipped - 38 lines] > > > > Peter > Sounds like you'll love the new reporting services in SQL Server 2005. > > Then the business can create adhoc reports to their heart's content. Business users aren't going to create adhoc reports. That would mean they'd actually have to do some work. SignatureAdam Machanic Pro SQL Server 2005, available now http://www.apress.com/book/bookDisplay.html?bID=457 -- Well, my opinion would be that business users are not going to create reports of any great use in report services 2005 report builder because it is simply way too limited a tool to be able to write anything remotely useful.......I am writing something useful with report services as a front end and I am already up to 14,000 lines of stored procedures....this is what it takes to produce the analysis that will make a difference....my current estimate is that I will go past 50,000 lines of stored procedures for the V1.0 product I am writing where the SPs do the work of 'making it useful' and report services is the presentation layer.... Sure, business users can write more reports (and I disagree with the notion that they would prefer to avoid doing some work..;-) ) but they will be the 'how many widgets did we sell last week' reports which tell you nothing about is that good/bad/otherwise and how to make a difference to the number of widgets sold. The business and IT worlds have voted. And the vote was business people insist that IT people write reports....and that right there has destroyed most of the opportunity for using information as a strategic weapon for competitive advantage because IT people rarely understand the business and business people even more rarely understand what it takes to build what is needed to use information as a strategic weapon......very, very few companies, as a percentage, have really been able to leverage their data assets to significantly increase profit of the company....and it seems almost no company is willing to do waht it takes to do so......and it seems business people are dis-interested in discussing this topic. We have, alas, snatched defeat from the jaws of victory on this one!! Peter www.peternolan.com Is it possible that 90% of that programming code could be replaced by an OLAP cube and a pivot table? > Well, > my opinion would be that business users are not going to create reports [quoted text clipped - 30 lines] > Peter > www.peternolan.com I have that question since 1998 when we started our first data mart project, and I am hoping SQL server 2005 can address this needs. Also in another thread, client / frond-end tools are discussed and ProClarity seem to be a perfect companion with SS 2005, which is additional cost. I wonder whether the BI Development Studio is there for compensating the lack of front-end tool, or for the 10% most complicated cases that cannot be solved by analysis service and reporting service? I am new to SQL server, so I need this answer. Thanks in advance. The BI Development Studio is what you use to create Analysis Services cubes and Reporting Services reports (and SSIS packages) it is not an end user tool, it is a tool for developers. The main clients that MS provides for end users are Excel (including OWC), Reporting Services, the new Business Scorecard Manager and Data Analyser. Apart from Reporting Services these all require separate licensing (although most people have an MS Office license for Excel) If these tools do not meet your needs you will need to budget/plan for an additional third party end user client. SignatureRegards Darren Gosbell [MCSD] Blog: http://www.geekswithblogs.net/darrengosbell > I have that question since 1998 when we started our first data mart > project, and I am hoping SQL server 2005 can address this needs. [quoted text clipped - 5 lines] > reporting service? I am new to SQL server, so I need this answer. > Thanks in advance. Hi Dino, all BI tools talk to SQL Server very well......I do not know that ProClarity has any inherent advantage over anyone else in getting data from SQL Server.... BI Development Studio is there to be able to develop reports......Visual Studio is THE development tool going forward from MSFT and if reports are to be developed it was natural for a plug in to be added to VS. However the RDL generated is a public specification so you can expect lots and lots of other people to develop tools to create reports....MSFT will create a market for these tools I expect. There are well more than 10% of cases that cannot be solved by AS/RS....but AS/RS do give you a great foundation for most small to medium companies....in fact, I argue that just using the database engine and report services is also a really good foundation today........AS adds functionality, of course, but if you build the back end well, that additional functionality is not as much as might be imagined....or, put another way, a LOT of what people do in AS can be delivered from the underlying database making data available sooner and in a more scalable environment..... Peter www.peternolan.com This has been a very interesting thread. First, I think a data warehouse guy must understand reporting, but like an academic, doesn't necessarily have to do it. You must, however, understand all the data from your source systems and whether it is useful for report and analysis. It is true that BI Dev Studio is a developer tool, but the new version (SQL 2005) is soooo much better than the last, that some tech-savvy end users can be taught to use it (I've done that with great success). It doesn't eliminate the need to IT report writers, but it has cut down on requests quite a bit. The more interesting role of the data warehouse(/BI) guy is helping the end users to understand the new reality of their changing environment and to adapt to an analytic organizational culture. I have found that even my very behind the scenes techies are called on to assist helpless end users understand that their gut instinct was wrong, because we know with certainty that the data is right. Sometimes they are even asked to explain in gory detail the transformation the data goes through as it moves to the data warehouse. Also, Microsoft has bought ProClarity to address the obvious gap in their BI solution. I have been told by insiders that ProClarity should be integrated into the product suite and offered after the MS fiscal year begins in July. Waiting impatiently for that!! Tricia Wilcox Almas BI Product Manager Compass Technology tricia.almas@compass.net Hi Tricia, I think the most interesting job of a 'BU Guy' is transforming the profitability of a company...but it seems not many people are actually interesting in talking about that.. Peter everything can be solved throgh analysis services learn MDX: it is an awesome language.. but definitely defintiey definitely-- a pita to learn -aaron > I have that question since 1998 when we started our first data mart > project, and I am hoping SQL server 2005 can address this needs. [quoted text clipped - 5 lines] > reporting service? I am new to SQL server, so I need this answer. > Thanks in advance. Hi Aaron, "For some reason, lots and lots of people have come to believe that cubes are the answer to all BI questions" Well, I guess you are one of these lots and lots of people... :-) Best Regards Peter Peter it is the best technology for 99.999 percent of the reports that you guys write by hand let the end users write their own reports; instead of calling you and saying 'can you get me those 2006 numbers yet'? Hi Aaron, and your research to prove that 99.999 percent of all reports can be done through cubes is...where exactly??? The 15,000 companies using business objects might disagree with you, along with the 5,000 or so using microstrategy...nary a cube in site in either product.. ;-) The facts are there is a place in the work for cubes and a place in the world for query databases directly......cubes have some significant limitations which are constantly being rolled back....but they are still there... Peter Business Objects is roadkill brotha i disagree; cubes have no practical limits; relational queries? they take HOURS AND HOURS AND HOURS to run Hi Aaron, Business Objects is roadkill valued at USD2.5B as a company.....not bad..... If you believe cubes have no practical limits it would merely indicate that you haven't seen them for yourself yet....not that they do not exist. And developed properly relational queries do not take hours and hours to run, they take sub second or maybe a second or two when the query is large.....for very complex 10 seconds should be enough....those are pretty much the guidelines I have been working to for the last 14 years...of course, when the query must do a large amount of sorting and summarising it may take longer.....but long running queries for a large percentage of queries is a symtom of bad design. I did a project 5 years ago with a 2cpu windows server running sql server 7 with 50M rows and 80M rows in my two largest fact tables, we had 100 BO users and report presentation time was pretty much subsecond......we ran and cached the reports..... Most queries against the various summaries were under the 3 second mark... Those are pretty amazing stats such such a small box... And yes, we did talk to MSFT who said 'put everything in a cube' but when we asked them to guarantee that it would work they took the line they do not make guarantees...indeed MSFT chose not to provide any written guidance on sql server 7.0 performance given even the most specific details and so I was paid a weeks consulting time to prove sql server 7 could handle the load....and I was an Informix employee at the time...LOL!!! So relational databases can be made to be very effective and very fast in decisions support.....and provide a host of features that the cube products do not..... That said, both have a place in the world...... Best Regards Peter Nolan www.peternolan.com > Business Objects is roadkill brotha > > i disagree; cubes have no practical limits; relational queries? they > take HOURS AND HOURS AND HOURS to run And cubes take hours and hours to build. Actually Microstrategy does build it's own proprietary cubes. Business Objects is commonly used for either analysis from cubes or a relational database. > Hi Aaron, > and your research to prove that 99.999 percent of all reports can be [quoted text clipped - 10 lines] > > Peter JT, well, a little misleading.... As I understand it, microstrategy can now be configure to build an internal style cube and this has been a rather later addition to add some of the cube style features to MSTR...this is not a requirement...but i have not been dealing with MSFT for quite some time.....BO has really taken a lead in that space. Of course BO and many other tools read cubes...there are lots of cubes out there to read....Cognos Powerplay web edition was re-written to brilliant effect and is, for my money, the sexiest BI interface out there today...entirely dependent on cubes......I've done a few projects with the new web version and I like it a lot..... Best Regards Peter Nolan www.peternolan.com > Peter > [quoted text clipped - 3 lines] > let the end users write their own reports; instead of calling you and > saying 'can you get me those 2006 numbers yet'? You're living in dreamland if you think that cubes will answer everything. Hi JT, if 90% if these SPs could be replaced by cubes I would be doing it in cubes.... ;-) For some reason, lots and lots of people have come to believe that cubes are the answer to all BI questions......not the least cause of which is sales reps from cube vendors telling people so....alas, this is just not true. Cubes have their advantages and disadvantages just like every other technology. In version 1.0 of this product we are doing no cubes. We can do lots and lots of useful and valuable things without the need for the added complexity, delay and cost of cubes. And since we have all the data we need in tables we can easily get them into cubes if we want to.... But some of the great features of cubes like multiple levels of summaries we have been doing in databases since the early 90s........ Sure, we will miss out on some of the features of cubes....but it's a trade off like anything else.... Cubes and data mining are on the list of things to do later... :-) Peter www.peternolan.com cubes are the most important thing in the world screw static reports and tsql > Hi JT, > if 90% if these SPs could be replaced by cubes I would be doing it in [quoted text clipped - 21 lines] > Peter > www.peternolan.com It sounds like you were working in an IT department with a "if it aint broke then don't fix it" mentality, and if there is some poor guy willing to spend 50 hours per week hacking out SQL queries to fullfill their requests, then as far as they are concerned, the system aint broke. Of course, when the poor guy gets pushed to the breaking point and turns in his resignation, they soon discover just how broke their system really is. Ideally, there is no reason for end users to be constantly requesting new queries to be written. A well designed MS Analysis Services cube with a simple pivot table or dashboard front should be able to accomodate most of their requests. Also, they require enough creative design to keep the developer interested. >A while back, I had the worst 6 month period of my life. It was awful, >every [quoted text clipped - 18 lines] > > TIA, ChrisR. |
Reply to this Message |
 Sign In
 Join
 My Latest Posts My Monitored Threads My Blog My Photo Gallery My Profile My Homepage Start New Thread Enable EMail Alerts Rate this Thread
|
©2008 Advenet LLC Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.