For years I’ve been using excel or dynamic SQL to generate SQL scripts to do data loads and updates and to generate IN predicates . For a small number of rows or values this is quicker than importing the required data into SQL Server. A few days ago I idly wondered how difficult it would be to create a SQL Server Management Studio extension to do these jobs directly in SSMS. The answer was quite hard but the results are quite satisfying. Continue reading “Create a SQL Server Management Studio Extension”
Developing a Salesforce Download
About three years ago I was working on a Salesforce implementation for a campaigning charity. My role was to develop a SQL Server Operational Data Store that mirrored the data in their Salesforce instance. After considering a number of options we decided to develop our own download interface to regularly refresh the SQL Server database from Salesforce. Initially this was developed using Talend as the middleware, which worked pretty well for daily incremental downloads of changed Salesforce records. Continue reading “Developing a Salesforce Download”
Using the Google Maps Distance Matrix API in .Net
Knowing the distance between your staff and students’ locations and your institution has innumerable uses. From timetable planning to calculating the CO2 impact of your organisation. Given a person’s postcode calculating their straight line distance from an address is straightforward but not that useful. Actual driving, cycling or walking distances would be better. Here the Google Maps APIs can help. Continue reading “Using the Google Maps Distance Matrix API in .Net”
Converting UTC to Local Time in SQL Server
For Salesforce UTC Date Fields
Salesforce stores all datetime fields in UTC (Coordinated Universal Time) and converts them to the users’ local time when they’re displayed in the UI. If you’re extracting date data from Salesforce via the APIs then they’ll also come across as stored – so in UTC. If you want them to be in local time you need to convert them back. Fortunately this straightforward in SQL Server. Continue reading “Converting UTC to Local Time in SQL Server”
Auto Create Views from the Tribal SITS data dictionary
As I’ve discussed the Tribal SITS Data Model does not have the nicest table and fields names and has a huge number of lookup tables. In this situation I like to build a layer of views over the base tables to provide a friendlier interface to build queries on. Still this is a bunch of work – could I auto create these views from the SITS data dictionary? Short answer: Yes. But they’ll probably need a bit of manual tidying up. That said I can produce working views that will still save a lot of effort. Continue reading “Auto Create Views from the Tribal SITS data dictionary”
Navigating the Tribal SITS Data Model
The Tribal SITS database doesn’t have the worst data model I’ve ever used but it’s still not the easiest to work with. The table and column names are logically organised but most are just four characters long. So they’re effectively codes the meaning of which is not always clear. I typically build a view layer for the tables I use frequently to add in more verbose table and column names as well as adding in all the lookup tables of which there are hundreds. Another issue is there are no foreign key constraints on the database. Aside from the fact that referential integrity is not enforced this makes finding table relationships difficult. On the up side SITS does maintain its own data dictionary which helps with these and some other issues. Continue reading “Navigating the Tribal SITS Data Model”
Reporting Timetable Exceptions in Power BI – Student Hours per Day
Having defined a timetable fact table and a timetable calendar dimension in previous articles I’ll now consider a particular reporting use case. I want to identify students who are timetabled for more than a certain number of hours per day. I want two measures here. The number of students who exceed X hours teaching in any single day and the number of days where each student exceeds this total. Using a timetable like this I can summarise the data at student / day level in SQL and add it to the Power BI model or create it in DAX as a calculated table. This will let me define the measures I want. Continue reading “Reporting Timetable Exceptions in Power BI – Student Hours per Day”
A Timetabling Academic Year Calendar
There are loads of good scripts out there for creating calendar tables so I’ve never felt the need to put up my own. But since I’ve written an article that references the need for one I’ll chuck my own into the mix as well. This will be a timetabling academic year calendar. So rather than just dates it will go down to the smallest timetableable unit – 15 minutes – so each row will be a 15 minute slot. Continue reading “A Timetabling Academic Year Calendar”
Creating a Timetable Fact Table from a Schedule
In general when designing a fact table I’d always aim for the most detailed level of data (lowest grain) possible. It’s always easier to roll up data than to increase the detail level of data later so best to include it up front. For timetabling data this may mean expanding a very compact schedule record to vastly more rows. But even for millions of rows it’s still worth going as detailed as possible to make the reporting easier. Continue reading “Creating a Timetable Fact Table from a Schedule”
National Student Survey Data in Power BI
The national student survey results are returned as a set of Excel workbooks that breakdown the responses by a variety of criteria – each workbook providing a different breakdown. By course, by age etc. This is fine but the structure makes it difficult to visualise the data in other tools and to compare the NSS metrics with other metrics about a category. Continue reading “National Student Survey Data in Power BI”