KMyMoney Transaction Matcher Ace Jones <acejones@users.sf.net> Use Cases Case #1A: Matching hand-entered transactions manually I enter a transaction by hand, with payee, amount, date & category. I download that transaction via OFX. Now, I have a problem. There are two transactions in my register, when there was only one in real life. If I delete the OFX transaction, the bank ID for this transaction is lost, so the next time I download via OFX I will get this transaction again. Using the manual transaction matcher, I can identify that these are the same. This will keep the payee and category I assigned by hand, but retain the bank ID of the downloaded transaction, avoiding the download again problem. If the date or amount don't match, it will use the downloaded date & amount. As a refinement, this behaviour can be globally configurable, or even prompted for each transaction. Case #1B: Matching only one side of a split Similar to Case #1A I have a checking account CH, and a credit card, CCA. I download a statement for CCA. I assign the category on the payment as a transfer from CH. Then I download the statement for CH. Now I have the same problem as Case #1A. This points out that bank ID's actually need to be a property of the split, not the transaction... because when this transaction is fully resolved, it will have two bank ID's: The one from CCA and the one from CH. Case #1C: Resolving conflicts in single-split matching Same as Case #1B, except I download the CH statement first, and I have 2 credit cards. Now I have another problem. I have a checking account CH, and two credit cards, CCA and CCB. When I download my statement from CH, the transaction payee for the payments to both CCA and CCB are CREDIT CARD PAYMENT.
The last time I entered a payment in CH, it was my payment for CCB. Category matching (see Case #3) will assign the other side of BOTH transactions to CCB. Later I will download the statement from CCA. Category matching will match the other side of that transaction to CH. Thus, in CH, I am left with two transactions, of the same amount and date. One is to CCA (the correct one), another is to CCB (incorrect). Somehow, I need to (manually?) match these two transactions into a single transfer from CH to CCA, with both the Bank ID from CCH, and the bank ID from CCA. Case #1D: Matching hand-entered transactions automatically Same as Case #1A, except the automatic transaction matcher will see that the amounts are the same, the dates similar, and match them for me. However, if I don't like the matching, I can unmatch them manually. This will leave 2 transactions. The one exactly as I entered it, and the second, exactly as I would have downloaded it without matching. Case #2A: Payee matching I shop for groceries at Whole Foods Market. When I download my grocery transactions, the payee is set to WHOLE FOODS MARKET 1000 ACE ROAD 800 555 1212. The first time I download one of these transactions, I change the payee to Whole Foods Market. The next time I download one, the payee is already set to Whole Foods Market because the program remembered the change. Case #2B: Advanced Payee Matching My bank has the insidious habit of inserting variable information into my payees, thus thwarting the logic that enables Case #2A. It may insert the date of payment, or an invoice number, or transaction number or something else. e.g. JUL 18 0846 SOUTHGATE HARDWAR PAYSON UT I would like a way to say, anything that has SOUTHGATE HARDWAR in it is in fact Southgate Hardware. I'm not entirely sure what the good solution is for this. One proposed solution is to have a matching regular expression in the payee. However, while I might find this useful, asking our target user to master regexp's is too much. I think a sub string approach is best, but I'm not sure how to enable the user to enter it. Perhaps just a match key field
in the payee. Case #3: Category matching The most recent transaction involving Whole Foods Market was set to the category Groceries. When I download another transaction which matches the payee to Whole Foods Market, the category is set automatically to Groceries. Case #4: Scheduled transaction matching We should also consider the uses cases of matching against scheduled transactions. I'll save that for a future date. This cases I can think about right now are: First, you enter the scheduled transaction and then LATER download its match. Second, you download the matching transaction, decline to match it with it's scheduled transaction and then later try to enter the scheduled transaction. Ideally, the scheduled transaction enterer detects the match and prompts the user, "It looks like you already have this, continue anyway?" or some such. MS Money's Implementation I've been using OFX based home banking in MS Money for years from numerous banks on even more numerous accounts and I've found it the perfect balance of speed and accuracy to enter transactions. When transactions are downloaded (OFX, HBCI, QIF, or anything else), they are entered and marked as 'downloaded' with a really obvious formatting change (bold text in Money's case). An attempt is done to match transactions automatically (Case #1D), and if a transaction is matched, there's a note that "this transaction was matched with an existing transaction" along with a button to UN match it. Matching requires 2 transactions: One entered by the user, and the other one downloaded. If a match is found, the 2 transactions are DISPLAYED as one transaction, but the engine maintains them as two separate transactions. That's needed so the user can say "No thank you, those do not match," and the transactions will be separated. They are merged into one when the user accepts the downloaded transaction. You have to 'accept' each downloaded transaction, whether or not it was matched. There is an 'accept all' button too. There is a button to manually match transactions (Case #1A), or unmatch incorrectly matched transactions.
The highlighted transaction in Screen #1 was automatically matched, or merged in Money's parlance. This fact is spelled out in plain language in the transaction display itself, and the details of both my hand entered transaction and the bank's version are shown. As Tom pointed out, the (rows per transaction) varies between regular and downloaded transactions. This will get weird coding wise... but we can use a different representation anyway ; ) This screen shows a similar screen to #1, but the transaction forms are turned on. Note that the ledger view automatically goes into "one line" mode when you're showing the forms, and also that the variable line bit Tom mentioned is still there.
Screen #3 shows the manual matching screen. This screen is brought up by pressing the Change button on the highlighted transaction. From here, the user can declare that this transaction is not to be matched with an existing on, or to change the one it was matched with. Screen #4 shows what Screen #1 looks like after I've accepted the matched transaction (highlighted). If you download a transaction that has not already been entered, there is no match. So the single displayed transaction is the downloaded one (with an altered payee and/or category as noted earlier). When you accept it, you enter it in. In practice, in MS Money, that transaction is still a 'live' transaction even if you haven't accepted it, e.g. it shows up in
reports, etc. Whenever you go back to your register, it's sitting there bold waiting for you. If it couldn't complete the split based on past data, it assigns it to the "Unassigned" category, which is also how it shows up in reports. (I'm not sure I love the "Unassigned" category instead of just doing what we do now. But it's probably a bit more user friendly.) Screen #2 shows a transaction which was not merged, because it wasn't already entered in the ledger. Here the user still has to accept the transaction. Pressing change will bring up the matching interface (Screen #3), because perhaps I do want to manually match this with another transaction. This is my last chance to do so. Once I accept the transaction, I can never match this again, and the transaction becomes just like any other transaction in the system. Note in Screen #4, it doesn't say This transaction was downloaded from your financial institution any more. This is in fact the normal use case for me. I RARELY hand enter transactions, preferring instead to download them all from the various banks. So there is rarely a match. The exception is usually for scheduled payments. Money is also aware of your scheduled transactions even if you haven't entered them. So it will attempt to 'merge' with those for you, in effect saying "This looks like it's your mortgage payment, even though you haven't entered it." Again, you can say, "No thank you, that is NOT my mortgage payment." MS Money clearly handles Case #2A (Payee Matching). When you manually match, it seems to update its payee mapping. Everything described here is done within the regular register itself, without a transaction list dialog like KImportVerifyDlg. That's handy, because sometimes you want to flip back and forth to payees or categories when you're dealing with downloaded transactions.
Case #3 is handled cleanly as well (Category Matching). Some months I won't shop anywhere new, so I can download and then 'accept all', and I'm totally DONE. Nice! Implementation Transaction Matching Logic In order to match an existing transaction, an imported transaction must have the exact same net amount. It must be within +/ 3 days of the existing transaction. Ledger Changes Unlike MS Money, we will not show both the original and matched information in the ledger. The ledger shows just the resulting transaction, after the matcher has made any changes to it. Imported transactions will have the yellow background color that they currently do only in the KimportVerifyDlg. In multi line view, they have the text IMPORTED in the bottom line of the number field (which will cause the number field to be wider sometimes), and two icons in the bottom line of the Date field. One icon is the green check box to Ok the match, and the other is the wrench to Match something more appropriate. See the above screen shot. This information is also conveyed on the button bar when transaction forms are enabled. To accommodate this, perhaps transactions with the imported flag are always shown in multi line mode, regardless of the ledger setting. The Match Dialog The things you can do in the match dialog are:
See if the matcher matched an already existing transaction. Choose to un match those. Choose to manually match the imported transaction with a different existing transaction. See if the matcher changed the date, and choose to keep the old date. See if the matcher changed the payee, and choose to keep the old one. See if the matcher changed the amount, and choose to keep the old one. This would only happen if you manually matched it to another of a different amount. Category Matching No special UI is needed for category matching. To some extent this is already working. It could be cleaned up a little. Payee Matching No special UI is needed for Case #2A payee matching. The way I imagined this working is storing a payee match map in the file, mapping all known imported payees to known payees.