NOTE: This page is probably very out of date. The current english GnuCash wiki is on http://wiki.gnucash.org/wiki , online since December 2005. If you want to contribute to any of the discussions below, please move the content from here to http://wiki.gnucash.org/wiki and continue the discussion there.

Summaries of subjects that have been discussed on the mailing lists:

Contents

---

1. Generic-Exporter

On the gnucash-de mailing list the idea of a generic-exporter has developed to improve integration of the HBCI-protocol features and facilitate many other features.

A generic-exporter framework will also be good for other online protocols, file formats, check printing, data syncing etc.

Right now generation of hbci online transactions can only be done in a stand alone hbci-gui-module. Such nice features as scheduled transactions can't be used because they are not hbci-aware. Without a exporter concept the same would be true with any other interface implementation.

The concept involves defining a format for "bank contact" data that is stored within the description fields of regular gnucash transactions which should not be a problem for any other feature to handle (passively).

The exporter would have to

1) look for transactions that are pending "online actions",
2) read the "bank contact" and info data,
3) hand those transactions to specific hbci-, ofx-, web-, print- interface-exporter etc.
4) and change the status of transactions accordingly

Pending "online actions" would be characterized be two things. The "action" field is set to something the exporter recognizes like "wire transfer" and the status is set to a yet to define new status like "s" for "send it".

One problem is that bank interfaces need datafields that are not explicitly necessary in gnucash. The account number, the bank, and the name of the receiver for example.

Therefore "bank contact" data is stored in the description fields in a unique way. The new International Bank Account Number Standard (IBAN) is suggested to be used.

Here is an example string of a description field:
electricity "customer #999" <local-electic@DE00 5555 5555 1234 5678 90>

The bank-contact data is written in brackets. The left part is the name and the right part is the IBAN. The first two letters from the IBAN specify the country, next is a two digit checksum, and for German IBANs (DExx ...) a 8 digit bank code and a 10 digit account number follow. Here "customer #999" is explicitly specified in quotes for the info field of the online order, so that the rest of gnucash's info field does not get exported and remains as a private note. Otherwise the whole rest (not in brackets) of the description is used as info field for the online order.

The generic-exporter could also display the transactions for verification before handing them over to the interface specific exporter, maybe in the (printable) International Payment Instruction (IPI) standard form.

PS: To support international transfers it be allowed to append the Bank Identification Code (BIC ) to the IBAN. So that a complete bank contact would read: <Name@IBAN.BIC>

A lot of this can be achieved in GnuCash/QsfXml - this provides a generic export and import function in plain XML for selected entities from GnuCash (and other applications). QSF is part of the Gnome2 port of GnuCash.

---

2. Accounting Periods

If a user says he needs to close the books in his accounting program he doesn't necessarily mean he needs to cut off all previous transaction data. He probably would not want that at all when he hears about the consequences, but would rather just like a special report for the previous period (whatever length it had) and "reset" the date from when some current balances are calculated (for example "year to date")

2.1. Book Closing(data splitting) vs. Accounting Periods

Two general approaches to arbitrary periodizing:

2.1.1. (A) description of "book closing" (aka Plan A from /src/doc/books.txt)

The program partitions the sum total of accounting data into a multiple sets; each set is referred to as a "book". The goal of book-closing is to "freeze" older data, so that it can no longer be (accidentally) modified, as well as weed out older data from various displays/reports (e.g. not showing closed accounts). In addition, if different books are stored as distinct files, then startup performance (the loading of the data) can be greatly improved. The down-side of using multiple files for multiple books is that generating multi-year reports becomes harder. This is a particular concern if books are closed monthly.

The natural boundary for a book closing is an "accounting period"
. The current code in GnuCash for book-closing treats the data is if it were the end of an accounting period. Bank, Asset, and Liability account balances are carried forward into the new book/period. The opening balances of these accounts are balanced with equal entries in the Equity account (using plain-old ordinary GnuCash transactions between the Equity account and the asset accounts to designate the opening balances.) The balances on Income and Expense accounts are trivially zeroed (income/expenses can never cross the boundary of an accounting period).

The handling of stock accounts or inventory is trickier: gains and losses on sales and purchases cannot be computed if the share balance of a stock account is merely brought forward. In this case, shares must be grouped into lots, and all open lots, no matter how old, are pulled into the currently open accounting period (book).

In this "book closing" (aka Plan A from /src/doc/books.txt)

• The program generates multiple arbitrary books (files) corresponding to accounting periods. Additionally this must not break reports spanning multiple books and periods. And importantly the concept also needs to be able to neutralize assumptions that where made in periodizing the accounting data (depreciation, amortization etc.) for other reports and assumptions.

2.1.2. (B) "accounting periods"

• In this approach all reports including the classical balance sheet (I hope that is the correct term) are generated from the mere transaction data optionally merged with a set of "closing transactions" for the particular period. Those "closing transactions" (describing what to account for where and when!) represent the assumed depreciations, amortization etc. for a given period and accounting standard/paradigm. (i.e. us-GAAP, German-GoB etc.) Alternative (B) is preferred, because it is much more flexible, and only it can accommodate different regulations or accounting principles even dynamically. A great plus for internationally employed software. Besides many things in GC (reports, overview bar) already work in such a way.

2.2. Distinguishing Accounting Periods and Data splitting

The problem of too large files or databases is a different for periodizing in accounting and should be handled by a separate "file splitting" function in Gnucash.

To reduce the size of the file it would be desired to split off data preceding an arbitrary date. (maybe older than 5 years or so) You would end up with two files, one just like you had it on the given date and a current one starting at that date with starting balances accordingly.

It should be possible to have different splitting dates for different accounts. This way you do not need to split off your long time savings accounts with only a couple of transactions per year for example. This way you can keep track of it (generate reports). Some accounts do not have that many transactions that would make them worth to time-split them anyway.

Fortunately reading the "book closing beta 2" announcement it seems a data splitting function is pretty much already there even though it more or less tried to implement periodizing as described in (A).

In "ANNOUNCE: Book Closing Beta 2" Linas Vepstas described the current "book closing" implementation which mixes periodizing and data splitting:

As a side-effect of book closing, capital gains (losses) will be
automatically computed using a FIFO accounting method.  Stocks
or other non-cash assets that haven't been sold will remain in the
currently open book, even if the purchasing transactions were made
a long time ago.

The biggest drawback to the automatic computation of the gains/losses
is that there is no GUI to specify alternate accounting methods
(e.g. LIFO, or hand-picked lots, etc.)  There is basically no practical
way to 'undo' the results of the gain/loss computations.

... or to do other kinds of computations after "closing a book" in this sense, because periodizing assumptions would be mixed (hardcoded) into the data.

2.3. Advantages of accounting periods

• Saving separated period definitions (with own opening/closing transactions) for example "2003-us-GAAP" "2003q4-us-GAAP" or "2003-custom" thus allowing multiple and overlapping periods has several advantages:

  • Different types of periods with "closing transaction" applied like "patches" can represent different sets of assumptions and be compared. Different accounting standards and switching between them can be supported. Lets say if you want to switch your view to us-GAAP the opening and closing transactions of type/name "*-us-GAAP" periods are taken into calculation and all reports will be based on just that. Or lets see how the reports would look like if we use the "German-GoB" or "us-ENRON-GAAP" assumptions with the same underlying operational transactions. Or don't consider any "closing transactions" and use the continuous cash flow view.

  • Allow optimizations: Stuart D. Gathman: What we do is generate "starting balances" at the beginning of an accounting period year. When printing reports (or displaying registers) that start with a given fiscal year, the starting balances are used instead of adding up all the transactions from the beginning of time. When printing a report that crosses a fiscal year boundary, the starting balances are ignored. As a further optimization, we keep summaries by period (as in week, month, etc, not fiscal year). So to get the tarting balance for the register display, you just need to get the starting balance for the fiscal year, then add up a small number of summaries for prior periods in the fiscal year, then any previous transactions in the current period. The summaries are updated whenever a transaction is posted.
  • Budgeting: Showing reports based on periods that are defined with future closing dates and estimated closing transactions (i.e. a "2004-budget" period with closing transactions and possibly "lot-ed" opening transactions if Budgeting needs to be done on a finer grained level than accounts).

2.4. Why is Book Closing/Accounting Periods Still Unfinished?

In fact, basic book closing & accounting periods do work "just fine" in the current CVS HEAD... as long as you are not using the GnuCash business features. It should be fairly straight-forward to enable the business features for book-closing, although there are still a few interesting conceptual issues involved. Also, the SQL backend doesn't really support book closing. Linas Vepstas is currently designing a new SQL backend (dubbed DWI) that should make it a lot easier to add new, arbitrary objects to GnuCash such as the business objects) and have operations such as book closing work "automatically" with the new objects. Progress is slow due to the evening/weekend nature of the development.

2.5. Things that would need to be done

2.5.1. YTD register view

• Allow the balance of income/expense account registers to be calculated as YTD sums or from an arbitrary date. Later allow to select an "accounting period" to display. Herbert Thoma will see if he can modify the "overview bar" (under the Toolbar) to display only the cash-flow after a certain date. (done)

2.5.2. Simple "Close Accounting Period" help druid

• At this point most of the reports (including the balance sheet) already have a start/end date to deal properly with accounts constantly growing. But any assumptions on depreciations, capital gains etc. have to be entered manually and explicitly in the form of as "closing transactions" currently in line with regular transactions. To help users closing a period (and learn about what things need to be done) a little Help-Assistant should guide the user through the steps necessary to determine those "closing transactions". (as a start it would only need to present help texts) The Assistant should explain that, why and how the following things may have to be considered with "closing transactions" for a period: For example, the step-by-step things to evaluate at the end of a German-GoB period are:
  • depreciation, amortization
  • changes in warehouse
  • periodical corrections of payments/receipts
  • mismatches in stocktaking
  • valuations (FIFO method etc.)
  • tax corrections
  The user will have to do a bunch of "closing transactions" according to his situation in order to make his balance sheet report compliant to regulations. A us-GAAP user will need to do a different set of things, but in any case he will also end up with a set of "closing transactions" for the period in order to obtain a balance sheet according to GAAP.

2.5.3. Implement accounting periods ("periods")

• Periods will have own "opening/closing transactions" associated to them. Then let the user optionally with the "Close Accounting Period" druid store separate opening and closing transactions and corresponding period information. (In whichever way provides best performance in file or DB backend) Implementation idea from Stuart D. Gathman: Date ranges for each accounting period are defined in a period table. This handles arbitrary irregular periods, short fiscal years (when changing start of fiscal year) - anything. Of course, a wizard can fill in date ranges for common periodizing schemes like 12 months, 26 biweekly periods, 13 4 week periods, quarters, etc. In addition, the period table records whether a period is "closed". This just means that no transactions can be added or changed in that period unless the flag is changed back to "open". This prevents accidents, and is essential to prevent corrupting history when many users are entering transactions.

To support periodizing schemes they would define at least a starting date and the length of periods but could also hold default depreciation methods or other info as template for future periods.

2.5.4. Data splitting function (from existing beta 2)

• As described above, Linas book closing beta 2 announcement already sounds pretty close, only period computations need to be removed, and support for account specific splitting dates might be added in the future. Referring to books when talking of gnucash files may be ok too. Only difference now is that book can hold more than just a single accounting period.

2.6. Conceptual Issues regarding an implementation

2.6.1. Accounting Periods

2.6.1.1. Unsolved

2.6.1.2. Resolved

• add 13-period support to Freq-Spec and the Freq-Spec widget. e.g. from the mailing list:

  One of the calenders my company uses is like this:
  13 periods
  1st period begins jan 1st, partial first week plus 4 weeks.
  2nd - 13th period begins on a Sunday every four weeks.
  13th period - 4th week may be less than full week because it ends on 12/31.

  • this is handled by the period table and periodizing schemes

• Another approach with 13 periods is to actually have two-level structure, i.e. distinguishing an accounting period (e.g. a week, a month, a quarter) and a financial year. In this case one may have 12 running monthly periods that start on the first of a month and run through till the last day of a month + an extra "year-end closing" period, that happens on the last day of financial year only. This is where you'd be closing annual profit and loss.

• Note also that having accounting periods does not mandate their closure automatic closure, nor does it require any capital gain calculations to be cast in stone, if the process is done on a regular basis. In fact, having accounting periods may make accounting work, actually, simpler by allowing prior-period adjustments to be tracked better. In this case until actual closure an accounting period change should simply mean a segregated storage for balances/turnovers of a given period, that only get updated if a posting is made with that period's code.

2.6.2. Data splitting

2.6.2.1. Unsolved

• The filename of the file with the previous, archived data should probably be saved in the KVP tree of the current data file. This need be done only from the file backend.
• Handling of lots in data splitting (was: book closing) is implemented but is poorly tested. Need to write test cases. Also test cases for prices in data splitting (was: book closing).
• price-saving in the SQL backend is probably broken, its certainly untested. Need to remove old deprecated price-lookup mechanism, and replace w/ qofquery.
• Not sure if the following still applies to new periodizing approach:
  • Need to mark archived data files as unalterable, and respect that markup. I think there's a 'closed' flag in the book, but I don't think its respected.
  • The book closing GUI (druid-acct-period.c) needs to save/restore period end date (the Freq-Spec) to KVP on open book. This would be easy once we have a freq-spec-to-kvp and freq-spec-from-kvp routines.

2.6.2.2. Resolved

• "data splitting" a long bygone accounting period should be avoided just like splitting unsold lots, but might be unavoidable if overlapping periods are defined. Splitting periods is special case to keep in mind for the "Data Splitting" function to in order to always provide correct margin values.
  • Split period definitions will probably need to be "busted" (stay only deactivated in the more recent file for possible later rejoining). This means the affected line of periods will start later into the file. In the older data file the last period ends earlier. Otherwise new assumptions for sub periods would be needed.
• Concerning different split dates for different accounts there is this little tricky thing I can't explain nicely, I certainly don't have the right programmers lingo for that. Maybe if it is needed Christian Stimming could translate it better. The solution sounds like this:
  • Even with different splitting dates for different accounts all starting balances will all have the same date. The date of the first account-split encountered. If an account is split at a later date only those transactions not touching any unsplit account get deleted in the current file. In every split-off account the starting balance from the specified split-date is offset by the remaining transactions when recalculating it for the common starting balance date.
• Have some way of remembering the quick-fill text from older books.
  • Is this really needed? Assuming the old data is more or less obsolete.

Side Question: Where is the importers transaction matching "learning experience" stored?

• Need to provide for loading of closed books, because this is needed for reports.
  • Should not be necessary separating data splitting and periods. The possibility to merge subsequent data files could be nice tough.
• Handling of multi-book reports ??? Need to work out the recommended way of making this happen....
  • Should not be necessary separating data splitting and periods. The possibility to merge subsequent data files could be nice tough.

---

3. The Concept of "Lots"

This chapter has been transferred to http://wiki.gnucash.org/wiki/Concept_of_Lots

---

4. Multi-Currency vs. Multi-Commodity (Inventory) Handling

This chapter has been transferred to http://wiki.gnucash.org/wiki/Inventory_Handling

---

5. The New/Improved DWI SQL Backend

The overall idea is to create a "generic" object infrastructure to map C objects (specifically Qof objects) to SQL tables in an "automatic" way. The mapping would be safe for multi-user, simultaneous updates (i.e. do the correct locking/versioning, etc).

The technology I've picked for this is, (ahem, excuse the favoritism), my own: a smash-up of qof and dwi (see SourceForge).

status & roadmap:

1) Use dwi to specify the mapping of qof object 'param' to sql table field. (Done)
2) Save & restore qof object to sql database, using above mapping. (Done)
3) provide versioning support, so that multi-user updates can be properly handled. (Started)
4) provide table locking (Started).
5) step back, look what else there remains to be done (e.g. figuring out how to support the business object COW).
6) Implement a proof-of-concept in 'gnotime', iron out the gotcha's there, before moving to gnucash.

The current demo code is in qof/dwi CVS only. the tarballs are way down-level). See dwi/examples/qof-proto which I have just barely started beating into the qof "backend" framework, which see qof/examples/backend. Good luck.

Not listed in the above:

-- a. DWI currently requires compile-time linking to gtk, even though there is absolutely no reason for this for the above. gtk is needed for other DWI features. This needs to be replaced by some kind of plugin/dynamic load stunt. This is technically hard/dirty; I'll reject patches that do this wrong.
-- b. test dwi with various other db's (I've only done postgres).
-- c. implement/flesh out the db login procedures.
-- d. change gnc-event to use qof entity pointers instead of guid pointers.
-- e. clean up core gnucash objects to use QofInstance. Figure out what else is common between all gnucash objects (e.g. COW ??), and add that to QofInstance.
-- f. "re-factor" price query, so that it uses the standard qof query mechanism instead of the hacky thing its doing now.

I really really need help with the letter tasks a.-f. and working on those will not put you on a collision course with where I wanted to go with things. (except possibly COW, which is potentially hard/paradigm breaking, so I'll be staring at that real hard). These tasks are also probably a lot easier than trying to grok the big-picture of the mainline qof backend development, especially if you are new to qof.

6. Online Banking

The subject of online banking has come up on multiple occasions on the devel list. While GnuCash supports HBCI connectivity, there is currently no support for OFX DirectConnect (think Quicken, MS Money), which most US (and UK?) banks use.

Several people have compiled a collection of information and utilities that should be useful for the eventual integration of DirectConnect into libofx and GnuCash, including a full list of OFX server URLs for all banks which support DirectConnect in Microsoft Money, and a Python script to download recent OFX transactions.

See http://www.jongsma.org/gc/ for more information.

Some other online banking features to consider (also applicable to HBCI, not just DirectConnect):

• Integrate online transactions more tightly with the account register to allow easier issuance of electronic payments.
• Merge all online communications into one interface (HBCI, OFX, OFC, etc) - a "Synchronize with banks" component. All online transactions created in a session can be queued up and than all sent at once when the user next synchronizes with their bank(s). This makes creating online banking transactions offline possible.
• Add an online_id slot for each split, rather than each transaction, making it easier to track inter-account transfers for two online banking accounts.
• Integrate scheduled transactions with online banking. This could allow both scheduling (recurring) electronic payments, and to reconciling imported transactions against upcoming scheduled transactions and marking them as paid.

For more details on these and several other online banking-related features, see http://www.jongsma.org/gc/wishlist.html

Update: Since version 1.8.10 Gnucash supports online banking (using AqBanking) via HBCI, OFX DirectConnect, YellowNet and DTAUS-discs (used in Germany) (MartinPreuss, 2006/03/08).

---

Back to German GnuCash Wiki Page