Wednesday, September 21, 2016

IFoA CA2 Sep16 Paper 1 - tips and strategies

Prepared this for my CA2 exams (today!!) and just thought I'd share it with everyone!

Paper 1:

Always clone original data and work on it on a "validation" sheet. Whether the question asks for it or not, always validate the data/model. Some checks may seem super obvious and hence stupid (I'm grabbing data from non-negative numbers, so the results, by the faith in excel, must be positive, why do I need to check for that?) but it's important. We need to understand if these results are as expected from the model or not.

Some simple checks include:
Min/Max/Mean/SD/Range checks
Reasonable checks e.g.: expect population to always be within 10% of previous years'
Context checks e.g.: Gender must be M/F, so the SUM of each COUNTIF should add up to data count

Have your results in 1 single sheet (usually called *Drum rolls* "results"). This is something I always miss out since I'm always in the zone of answering questions. Based on the answer scripts that I've read, it's a good practice. When you think about the end user, it makes sense too. Your boss doesn't really wanna go through the whole process. He needs to present to the CEO, he needs the answer- so make it easy for him (and the examiner) by consolidating everything in 1 sheet!

Similarly, all the assumptions used should be in 1 sheet (usually named "parameters"). To consolidate all the data and for the next user to easily identify what are the inputs of the sheet.


Plan. 15 minutes are given for reading and drafting. Make full use of it! Don't just jump straight into the excel sheet without seeing the complete picture. E.g. you may think that all the results should be under the methodology section but sometimes, it's more appropriate to include it under Data. It's important to know the outputs the examiner is looking for and find a suitable section to slot them under.

Regardless, I will share with you the format that I usually follow and some tips and tricks in drafting the audit trail:

Overview - this section is supposed to give the reader a summary of what the project is about. I usually use some text from the exam paper (after all, this was the document where I got a summary of what I'm supposed to do) and paraphrase.
What I do: I like to state the scenarios I'm modelling and what am I expecting to get out of the model. But I don't think that there are hard and fast rules to this section. Think of it as an executive summary.

Data - this section holds information on the data used and where it was obtained. It's kinda like Jenson's monthly actuarial email and he tells us that the actuarial data is from AS400. We learn from the best and hence we document where we get our data from too!
What I do: I usually just use the exam script and paraphrase. Noting to add additional information when necessary. When the data is unclean, I'll include a "data validation" section right here (since the question will more often than not ask for a plot of the raw data) and start explaining the clean up process. Make sure you include checks to ensure that there isn't any obvious errors! I like to plot my data to see the smoothness of the curve but I don't think they give marks for that.


Assumptions- where the assumptions are documented, usually in bullet points. If you run out of ideas, don't just type generic comments. Think of the model from these following angles:

Data integrity e.g. is the data correct and appropriate for use?
Statistical distribution Question will usually dictate the use of certain distributions; we need to question the validity (of the data), and suitability (of the parameters).
Time consistency For CA2, very simple assumptions are used. The decrement rate is usually fixed at a certain % or value. We need to add this assumption into the audit trail.
Independence/Correlation eg, drop out rates are independent of each other. Since it might be the case that when your best friend drops out of a course, you're likely to follow.
Delays This is applicable to almost every paper I've tried. The idea is that there is no lags between whatever you're modelling- could be payment and receiving of premiums, transporting woollen felts, improvement of dog's longevity etc. You get the idea.
Business mix If external data is used to model internal projects, we have to assume that the business mix will remain the same, or at least that the fluctuations are insignificant.
Shocks Assume that there will not be any shocks throughout the 5/10/20/25 years modelling period. Way off assumption, I know, but unless you wanna spend time adding a random element into your model (which you will not be given marks for), this is a necessary assumption.
Other factors: Allowance for interest, inflation, capital gains, better mortality? All these will affect the modelling results and should be stated before you do the modelling- to ensure you don't miss these effects out.
Brainstorm: what other real world factors can affect our model? Be the devil's advocate! A lot of these insights will be whispering at you when you're doing the modelling on excel so don't ignore these voices! Grab a piece of paper and write them down. There will always be some factors that are missing out; if not actuaries will be able to predict the future ^^

Methodology- to document the method and approach used in our model. One issue I tend to face is that I don't know the level of detail required. Am I supposed to document how SUMPRODUCT and VLOOKUP works? I don't think so (since the reader is assumed to be actuarially trained). But all the other calculations are kinda... so obvious to understand from the excel sheet no?

Ah hah! This is why we need CA2. What tends to be most obvious for the modeller may be really difficult for someone who's new to the model to understand. This is when we need to take a step back and try to understand it from another person's point of view (it's like role playing). Either that or make your model so well thought out (in the first 15 minutes remember?) that the explanations can be easily made.

One comment that I see a lot on the examiner's report is that :

"Many audit trails were too short and methodology sections often lacked detail.  The stronger candidates provided sufficient detail explaining their calculations with the very strongest also explaining why steps were being performed."

So details!!! details are important, as inconsequential you may think they are, 15 marks are given such that everything in the excel sheet is documented in the audit trail. This is also an incentive to make the spreadsheet as simple as possible!

Also, remember that 5 marks are given for signposting. Easy marks = free marks = YAS. You just need to ensure that you refer to the cells that you're talking about. This will help you and the reader be less confused about the model.

What I do: create sub-sections to document the excel sheet, from top to bottom, left to right. Just type in everything that you do and justify the methods. I only include formulae when necessary. (sometimes I'll use text from the additional guidance and paraphrase). I imagine trying to explain the excel sheet to a bunch of JC/Uni students instead of fellow actuaries.

Make sure you choose the appropriate graph too! A rule of thumb that I follow is to use
the line graph when there is a time series and a bar graph when we need to compare. If it's a time series that you need to compare, stick to the line graph :)

Some checks to include at the end of the model, after the results are shown:
Summary statistics. The simplest form of checks. Ensure that the data count is correct and the mean makes sense.
Reconciliation of models. If you have a model that was based on another model, enter the first model's parameters into the second model and see if the results are the same. This ensures that the logic in both models are similar and hence comparable.
Shape of graph- talk abit about the shape of the graph and why it looks reasonable.
Impact analysis- talk abit about why the impact of different models and how your results can be expected. e.g. passing rate has increased, hence the number of passes is higher. (I know this sounds like a tautology and I used to hate it and it made me so angry! But I guess I got over it.. take some time and meditate and you'll feel better about the whole situation..^^)
Differences - highlight any significant differences in results and try to make sense of it.



Time management. This is the strategy that I tell myself to follow but sometimes I get carried away by modelling (cause we all love excel so much). But it's a good guide as to where you should be at each time period.
first 15 min - read and plan, try not to get your hands dirty. Understand the input and output of the model and what the examiner requires in the report.
next 1 hour - excel. model model model. Remember to include checks!
next 1.5 hour - start drafting the audit trail. You'll realise that some steps taken in the previous hour may not be the best way to do things and will need some time to remodel and redocument it. If you did sufficient planning, this will happen less frequently. Trust me, I've done many rounds of timed sittings. 
Last 25 min - take 1 minute to meditate and clear your mind and relook at the model and audit trail from a third person's point of view. This is the time when you look at your work and ask yourself: "what the hell was I thinking then?". Also, make sure to correct the voice of your report. I tend to write things like:

"we calculated the persistency rates by taking 1 - mortality rate "

when the audit trail is meant to be written in a passive voice  i.e.:

"The persistency rates are calculated using the formula {1 - mortality rate}".

I also take this time to prettify my model/audit trail. Oh yes, ensure that your ARN is on the header/footer (I use the latter) of the report and excel sheet!
last 5 minute - upload! Please make sure you have enough time to upload the report!!! I have an email drafted to online_exams@actuaries.org.uk so that I can email them in case the portal doesn't work! Yes yes, I'm very well prepared tyvm ^^

Alright, that's about all the tricks I have in my bag! I'll probably write one tmrw for Paper 2. Here, I want to emphasize that there is no shame in copying and pasting. It'll actually save you a lot of your time.

Alright we are #5hoursout and I believe that the best tool I can bring into an exam is an alert mind- so it's time to nap. Good luck to everyone attempting the paper with me! May sept16 be kind to us all \(^_^)/ 

TLDR: Read, Plan, Model, Document, Check, Submit, all the best ^^

p.s. somehow blogger/my HTML messed up by bullet points and it's really ugly.. Sorry about the formatting. I know it's really messy. Take it as exhibit A on how not to present during CA2 ^^" 

? 11:51 AM