codefinance.training

Coding for Finance

C# for Financial Markets – an interview with Daniel J Duffy

C# for Financial Markets – an interview with Daniel J Duffy

C# for Financial Markets cover

Way back in 2013 we spoke with Daniel J. Duffy whose co-authored C# for Financial Markets with Andrea Germani. We talk about what makes C# a great choice for programming financial applications and how it contrasts with C++ in terms of productivity and flexibility.

C# for Financial Markets was published by ‎ Wiley on 18 January 2013. Learn More

Jacob Bettany: Thank you for joining us Daniel. You will be well known among many of our readers for your books and your background in C++, but for those who don’t know of you could you tell us a bit about your background?

Daniel Duffy: My background is in Mathematics, I have a PhD in partial differential equations. I once went to a conference in New York on Mathematics and Derivatives where I realised that a lot of the mathematics, and numerical mathematics in particular, that people were talking about using was stuff I did all those years ago in university and in real life later on when I went into industry, so I decided to see if I could apply some of the numerical techniques to Finance.

JB: Did you work as a practitioner?

DD: I did at first, I worked on one of the first option pricing software projects called RiskWatch for Algorithmics, this was a C++ library. Nowadays I deliver training for banks in C++, C# and Finite Difference methods along with some consultancy offline for certain banks as well, and I deliver in-house training courses on C#.

JB: From the Global Overview of your book, I see that you came up with the idea for writing it in a restaurant in Milan! Perhaps you could begin by telling us why you felt the book was necessary, and why you were well placed to author it?

DD: Well, I wanted to go to Milan to have a nice meal in a restaurant, that should be clear! My co-author Andrea Germani is a trader, we met when he attended one of my C++ courses in London and we started talking about maybe cooperating together writing some work on fixed income applications in Excel. I studied in Italy in the 70s and so I speak some Italian. We later discussed this in a restaurant in 2006 near the San Siro stadium in Milan. We decided not to do it in C++, because although it’s a wonderful language it’s very difficult to learn and something you really need to do every day if you want to become used to it. Some people say that you can never become an expert at C++, but that’s another story! So we decided on C# because it’s as powerful as C++ and easy to integrate with Excel and database systems and the so-called interoperability with all the other Microsoft products. So, seven years later in January 2013 the book was published. It’s based on my years of experience of designing software for systems, plus Andrea Germani’s trading input which comprises more than half the book. There were a lot of reasons that it took 7 years; people were busy, and C# was developing as well. It was good in a sense that we didn’t have the book ready because when the crash came in 2007 all the models needed to be changed as the Fixed Income models were no longer applicable anyway, so we spent a number of years getting new models up and running.

…in university people don’t in general learn how to develop, design and implement a system which remains flexible and doesn’t break down. It’s been like that for the last 40 years in computer science. It’s not suitable for real life. When you hit the ground running it’s a culture shock.

JB: What was it like collaborating and what were the main challenges when writing the book?

DD: There are always challenges with collaboration when you are at a distance of course. Aside from the changing environment, people are busy and there’s always big pressure on time and resources and so on. The challenges were in setting up new models but also learning a new technology in a new environment. So there were some quite technical challenges.

JB: Is it fair to say that the book is pretty cutting edge then, and completely up to date?

DD: Well, first of all, it’s the only book available in this area at the moment. It uses all of the advanced object-oriented programming techniques, all the other kinds of generic models in C# and the corresponding libraries. We also focus on designing the software itself so that it becomes flexible and easy to use. So in that sense it’s cutting edge. Having said that I suppose fixed income applications change and the models change as well but the basic ideas are always valid.

JB: Can you give us a brief overview of how the book is structured?

DD: It’s really holistic, we start off with an assumption that people don’t have any C# knowledge. The first 6 chapters are devoted to basic C# syntax and how to make classes, with some generic examples but also some examples which are more finance-related, relevant examples. As we progress we do much more advanced stuff both financially and also computer-wise, such as the financial models from Andrea featured after chapter 12. We also devoted a lot of time to integration with Excel and discuss things like multi programming and parallel processing. So it‘s fairly advanced with a number of new so-called finite difference methods, PDE models – Partial Differential Equations – and some new stuff is in there which I’ve developed and documented in programming with C# as well, so it’s fairly complete.

JB: You mention the parallelism, how well adapted is C# for dealing with that kind of environment?

DD: Some of the Fixed Income applications are fairly computationally intensive. It’s beneficial to be able to run some of these simulations in parallel, which is possible when the different tasks in your parallel program do not interfere with each other and you can run them all independently. C# has facilities for actually creating multi-threading applications so you get a number of threads and each thread is running its own job. That’s the basic multi-threading facility but C# has also got libraries for multi-threading. That multi-tasking makes the life of the developer even easier because the low level details which you normally encounter with the multi-threading have been masked and made easier for you so you just create your tasks, fire off your tasks and your parallel program just runs.

JB: Who are your target audience and what level of expertise is required for understanding the book?

DD: Well, readers will need to be familiar with at least one programming language. It’s not a book for dummies; the book presumes a certain modicum of programming experience. I think there are two main groups of readers. The first group of readers would be those with C++ experience and they can learn the basic C# syntax in about a week or so, it’s fairly easy because of the historical roots of the language I presume. There’s a lot of similarity between the two languages. The second large and important group would be DBA programmers and my experience is that they usually find the book fairly easy to understand particularly the sections regarding Excel integration. We also have an appendix on the fundamentals of object-oriented programming, this is always a major issue for people who have never used it and I feel that the only way to learn it is to learn from your mistakes and those of others.

It’s very easy to write applications which seamlessly integrate C#, Excel and the commonly used database platforms. Last week I wrote an application on the Monte Carlo method with C++ and C#. I reckon my productivity gains are 4:1 in favour of C#.

JB: C# seems to have grown in popularity in recent years, why do you think that is?

DD: I started off on C# in 2000, on a one day introduction course with Professor Eric Meyer from Microsoft. C# conforms to the ECMA standard which means anyone can customise it, and there is actually a version of C# on Linux, which I haven’t used myself. Basically it’s a Microsoft product and their premier platform to a certain extent. It experienced a slow growth and it’s only the last few years it’s become more popular.

It has advantages to other languages. C# is very suitable for desktop applications which run on Windows and work well with all the other Microsoft products. You can integrate your C# almost seamlessly with Excel so in that sense you’re a competitor to DBA, and to a lesser extent C++. It’s very easy to write applications which seamlessly integrate C#, Excel and the commonly used database platforms. Last week I wrote an application on the Monte Carlo method with C++ and C#. I reckon my productivity gains are 4:1 in favour of C#.

The language offers a trade-off between performance and simplicity. It may not be as fast as C++ for certain applications, but the question is do you need C++ performance?

JB: What appeals to you personally about C#?

DD: The time savings are an important aspect. The time and effort required to get a C++ application up running are not small and it takes time before you actually start getting results compared to C#, where on a very practical level you can put everything into one file which means you can be up and running pretty quickly. It’s a very flexible language. Not only does it support the object-orientated way of working but also new emerging paradigms, multi-paradigm languages. Also you see more and more functional programming languages such as Haskell, Camel and F# which is a functional programming language of Microsoft. Many of the features are coming into C# which makes software development even more applicable and appropriate. The last paradigm is called generic programming – these are called Templates in C++ basically and you can actually create generic code which satisfies many customer types. For example, one customer might prefer to work with double-precision numbers and the other might only want to work with integers. With C# you can have only one source code you need to maintain to cover both cases. Also all the different paradigms are integrated, so you can switch between them.

JB: What advantages does your co-author Andrea Germani find in C# during his day-to-day activities as a trader?

DD: I put that question to Andrea and he responded with the following:

1. The language offers a trade-off between performance and simplicity. It may not be as fast as C++ for certain applications, but the question is do you need C++ performance? Connecting to Excel is almost as easy as VBA. Time is money, and if you used C++ you’d be spending a week trying to get something working. The customer can analyse and deploy a solution very quickly. On the other hand, using the right design techniques the developer can create a well-structured software design using state-of-the-art programming solutions. You’ve got the best of both worlds.

2. D.NET contains the C# compiler. This framework has lots of libraries you can use so you don’t have to reinvent the wheel. The framework for multi-threading, parallel processing and also for SQL searching and selecting. The libraries are being continuously improved.

3. Very easy web and database connections. It’s very easy to connect to a MySQL database or SQL server from Microsoft, or in Oracle, just using standard interfaces which for trading applications is very important as well so in the future my own prediction is that the data is becoming more and more orderly. You see this a bit with the Big Data. Actually getting big data from a database which is something that has perhaps been neglected a bit the last few years. Because if you say as a Quant I suppose if you are making database systems people consider it as being inferior or something.

4. The advantages of the object-oriented programming which means you can encapsulate and put all your functionality into one place, into one class, into one module which you can reuse again and again. So compared to GPAs for example where your code is spread all over the place, it’s a mess, whereas in this case you can actually bring everything together and make it a bit more manageable.

5. The contrast to C++ in terms of memory management. In C++ you have to create your own memory and delete your own memory. Failing to do so will cause your program to crash, whereas with C# memory management is automatic. The runtime system cleans everything up and that’s extremely important for reliability and preventing your Excel sheet crashing. That’s similar to the Garbage Collector (GC) in Java.

JB: What kind of jobs does C# do best in financial environments?

DD: People write applications in many different languages. But let’s consider a programmer writing in C# alone – generally it is suitable for standalone desktop applications. In other words, applications which are meant to run on a single desktop. It may be getting data from other sources but effectively it is a desktop application running Excel. That is ideal for C# because of all the libraries it offers with opportunities for windowing, linking with other databases and it has a good performance. It may not be as fast as C++ but that’s not the point. .NET is a multi-language framework. In other words, I can write part of the application in C#, the other part is in C++ and there might be another part in Python. In other words, my existing C++ base I can integrate into C#.

JB: So it can be adapted to suit legacy systems.

DD: Yes, you have to create a wrap around the native C++ because it doesn’t speak .NET languages but it’s easy to create a wrapper for it. A general rule of thumb is to use C++ for the really calculation intensive part of your application and use C# for the rest.

JB: How about for web applications?

DD: I’ve got no experience really, C# is typically used only in Windows environment. Java would be better for web applications. Each language has its own strengths.

JB: What supplementary materials do readers receive with your book?

DD: There is a website, www.datasimfinancial.com where you can register on the forum using details in the book and you will get access to all the source code.

JB: So looking to the future now, to what extent are programming paradigms shifting and what in your view are the limitations of conventional methodologies as they are taught and deployed?

DD: The biggest bottleneck is the developers of course. In the sense that, the hardware is not so important as such, it’s that things are changing. A lot of developers are in the procedural frame of mind and that’s not very scalable. And there’s another group who are using the object oriented way of working which is also fine but it’s not the only way to work and I always think of the expression if you give somebody a hammer, everything’s a nail. We’ve got a solution for all problems. It doesn’t matter what the problem is, we’ll morph it so it fits into my solution. So a single paradigm way of working, only object oriented way of working, only design patterns, only procedural, whatever it’s not the way to go and the challenges are that the software is not progressing. The software languages and tools are all great but nobody can really use them yet. So if you look at the features coming up in C#, in particular the functional programming way of working (just working with functions, which is almost like mathematical programming), I would say that (my excuses to those who already use it of course) most developers do not think in this way.

JB: Is that a problem in education, that people are taught to think in one way rather than take a holistic view of the way these technologies and approaches interact?

DD: Yes, except for my courses of course! Well, if we take university education the problem is that people don’t learn to program in universities, it’s not valued as a skill so you get people with PhDs in Maths and Physics coming out of university and they start writing spaghetti code, just a mess. Because in university people don’t in general learn how to develop, design and implement a system which remains flexible and doesn’t break down. It’s been like that for the last 40 years in computer science. It’s not suitable for real life. When you hit the ground running it’s a culture shock. The focus in more on the widgets and the gadgets, all the nitty-gritty stuff rather than creating the architecture of a software system, design and implementation. So universities are way behind real life, in my opinion. In some of my courses I try to rectify this.

The software tools are certainly much better than, say, 15 years ago. The software languages have evolved, everything is fairly reliable, the hardware is good so you are actually able to write decent applications. There’s no lack of computing power but a lack of brain power in designing and implementing software applications. The future of the hardware doesn’t really matter. Behind the Microsoft interface you’ve got a bunch of GPUs.

JB: There’s been an explosion in terms of data, some might argue that the Big Data environment lends itself to better and bigger hardware solutions – tasks are computationally more demanding and the solutions could be better delivered in hardware than in software. Is that fair?

DD: I drive a 2.5 litre car, I just turn the wheel and off it goes. So for application developers it’s the functionality that they will get by default from the same interface, so that won’t make any difference. It is important in the sense that someone else has done it for you. So as soon as a new Microsoft or Intel interface to a bunch of GPUs comes along we’ll use it. But you have another group of developers – these are the guys who actually program the GPUs themselves or whatever. I had a demo once of the FGPAs available, that’s for most developers just too difficult.

JB: There’s a sense in which the technology is changing the whole environment for Finance generally, not only in terms of programming development but in terms of the actual shape of the market. Would you care to speculate on the extent to which banking is going to change because of technology over the next ten years?

DD: I don’t think it will change all that much – everything will be faster. I’m not part of the hardware world. Firstly it’s obsolete after two years anyway, secondly there are lots of resources to do this kind of stuff and you need to be in an environment where people actually use these machines. If you look at the population of developers in C# I would say 95% are just your desktop Quant VBA/Excel jockey and maybe 5% of these are supercomputer guys.

JB: Thank you very much for joining us Daniel, It was great to talk.

Comments are closed.