Building robust solutions, DFT (cant.) OLE DB destination, queries, source file, 563 TextFieldParser class, transformation tas

Transcription

1 Index ADO.NET connection manager editor, 90 description, 90.NET provider, 91 SqlClient connection manager editor, 92 source and destination adapters, 226 Analysis Services connection manager, Analysis Services Execute DDL task editor DDL page, 164 description, 163 modification, 164 Analysis Services Processing task editor description, 120 General page, processing settings page, 121 custom error configurations, 126 Dimension Key Errors tab, 125 object list section, processing options tab, 124 writeback table option, Analysis Services project description, 12 developer-friendly format, 13 folder structure, 13 query languages, 13 Analysis Services Scripting Language (ASSL), 165 Analysis Services sources, Auditing adding auditing to packages, 480 audit data, SSIS log table, 481 DFT, package with auditing calls, 480 batch-level AuditBatch table, 473 AuditTable, 474 description, 473 EndAuditBatch, 477 StartAuditBatch, SummaryTableData, batch/package auditing, logical view, 473 description, 472 enable, 472 package-level, 478 AuditPackage table, 478 description, 478 EndAuditPackage, 478 StartAuditPackage, Batch-level auditing AuditBatch table, 473 AuditTable, 474 description, 473 EndAuditBatch,477 StartAuditBatch, SummaryTableData, Breakpoints description, 159 events, Hit Count Type column, 161 Set Breakpoints options, 160 Building robust solutions accoutability custom logging, log providers, custom logging configuration, 576 package-level variable, 576 SQL statement, 577 description, DFT colum selection, 564 control flow pane, 563 data cotrols, destination editor, editor pane, error output, flat file destination, MessageBox,

2 Building robust solutions, DFT (cant.) OLE DB destination, queries, source file, 563 TextFieldParser class, transformation task, dynamism, 573 factors, 561 resilience data flow task, event handlers, Bulk copy program (bcp), 2 Bulk Insert task connection page, considerations, 127 description, 126 options page, Business Intelligence Development Studio (BIDS) Analysis Services project, 12 description, 12 developer-friendly format, 13 folder structure, 13 query languages, 13 data flow optimization, 521 description, 11 Import Analysis Services Database, 16 Integration Services CDATAsection, description, 18 designer window, DFT component, 28 key tags/properties, 21 package code view, package explorer, 24 project, 14 project files, properties, 30 Resolve References utility, SSIS toolbox, tool windows, undo and redo functionality, 19 Integration Services Project Wizard, 16 project elements, 12 Report Model project, Report Server project, Report Server Project Wizard, 15 Cache connection manager editor columns tab properties, 94 description, 92 general tab options, 93 Cache transformations colunrrnstab, connection manager page, 305 data flow, , differences, 308 general tab, 306 population, Common language runtime (CLR), 581 Conditional split transformations data flow, editor, input output selection, Connection managers ADO. NET, 90 description, 90.NET Provider, 91 SqlClient Connection Manager editor, 92 BIDS designer window, 84 cache columns tab properties, 94 description, 92 general tab options, 93 description, 83 DQS, 99 file flat file, types of, 87 FTP, HTTP description, 96 proxy page, 97 server page, 96 MSMQ,99 MSOLAPlOO, OLE DB description, 85 provider selection, 86 us. ODBC, 87 SMO,lOO SMTP, SQLMOBILE all page, 103 connection page,

3 - INDEX description, 101 properties, 103 SSIS connection manager dialog box listing, 85 WMI, Containers description, 157 Foreach Loop container, For Loop container, sequence container, 158 system variables, 339 Task Host controller, 202 Control flow Analysis Services Execute DDL task editor, 163 DDL page, 164 description, 163 modification, 164 Analysis Services Processing task editor description, 120 General page, processing settings page, breakpoints description, 159 events, Hit Count Type column, 161 Set Breakpoints options, 160 Bulk Insert task connection page, considerations, 127 description, 126 options page, containers, 107 description, 157 Foreach Loop container, For Loop container, Task Host controller, 202 sequence container, 158 Data Mining Query task editor, 165 description, 165 Mining Model tab, Outputtab, Query tab, Data Profiling task editor, 130 description, 130 General page, profile requests page, description, 107 DFT,110 executables, 107 Execute Package task editor description, 134 package page, parameter bindings page, Execute Process task editor description, 138 process page, 139 properties, Execute SQL task editor description, III expressions page, 118 General page, parameter mapping page, result set page, File System task editor description, 140 General page, 140 properties, 141 FTP task editor description, 141 file transfer page, General page, groups, Message Queue task editor description, 170 General page, Receive page, Send page, precedence constraints configurable properties, 156 description, 155 and parallel tasks, 157 types, 155 Script task editor description, 144 properties, script page, VSTA, script editor, Send Mail task editor mail page, properties, SSIS toolbox, Transfer Database task editor Databases page, 176 description, 175 properties, 177 Transfer Error Messages task editor description, 177 Messages page, properties,

4 Control flow (cant.) Transfer Jobs task editor description, 180 Jobs page, properties, Transfer Logins task editor description, 182 Logins page, 183 properties, Transfer Master Stored Procedures task editor description, 184 properties, Stored Procedures page, Transfer SQL Server Objects task editor description, 186 Objects page, 187 properties, Web Service task editor description, 149 General page, Input page, WMI Data Reader task editor, Event Watcher task editor, XML task editor description, 152 General page, properties, Data flow optimization columns list specification, 517 using execution trees BIDS editor, 521 SSIS execution tree, 518 SSIS Log Events window, merge join transformation, 516 parallelism implemention, performing complex preprocessing, 516 performing joins, rows limitation, 511 advantages, 514 conditional split transformation, 513 sales transaction table design, 512 source data, 514 security,517 sorting, 516 Data Flow task (DFT) control flow, 110 Hello World, 57 control flow pane, 58 data flow path, destination component, execution, mapping, source component, Real World census data table, 79 denormalized data query, derived column component, DER_SplitYearAndStatistics,79 description, 72 DFT_TransformCensusData,73 FF _SRC_Apportionment, 73 FF _SRC]opulationDensity, 73 merge join transformation editor, 76 ReaIWorld.dtsxexecution, 80 sort transformation editor, 75 unpivot transformation editor, Data flow transformations asynchronous, audit, blocking transformations, Business Intelligence transformations, 323 cache columnstab, connection manager page, 305 data flow, , differences, 308 general tab, 306 population, character map data flow, 254 edit, 255 mutual exclusive, operations, 256 conditional split data flow, editor, input output selection, copy column, data conversion edit, excel spreadsheet, SQL data types, derived column, export, handling error rows,

5 - INDEX high-level data view, SSIS, import column configuration, data flow, 262 data types, 263 input columns, 263 lookups advanced page, cache mode options, 299 case-sensitive, 299 columns page, connection page, data flow, General page, Merge transformation, Merge Join transformation, multicast transformation, nonblocking transformations, 248 OLE DB command connection managers tab, data flow, mappingstab, parameterization, 267 properties tab, partially blocking, 248 row count data flow, 319 editing variables, parameters and variables tab, rowset aggregate, percentage sampling transformation, pivot, row sampling, sort, unpivot, script component creation, data flow, data viewer option, 279 edit script, input columns, output columns, output path, 279 pop-up window, properties, review, synchronous, synchronous transformations, Union Ail, Data flow tuning. See Data flow optimization Data lineage Derived Column transformation, 485 description, 481 lineage and audit data, 486 LineageMetadata table, SSIS package, 484 Data Mining Extensions (DMX), 165 Data Mining Query task editor description, 165 Mining Model tab description, 165 properties, Output tab description, 168 properties, 169 Query tab description, 167 properties, 168 Data previews data viewer enabling data, fuzzy lookup output, 462 merge join transformations, 461 percentage sampling transformation, row sampling transformation, Data profile business keys, 428 candidate key approximate key, 447 column types, 447 primary key, 447 task editor, tolerances, data integration projects, 427 domains, 427 functional dependencies output, profile request, fuzzy grouping component data replacements, destination, 458 records, review results, 459 string data types, 458 fuzzy lookup advanced tab, 457 advantages,

6 Data profile, fuzzy lookup (cant.) column details, 453 configuration, 455 data flow task, 457 mapping columns, matches, 454 queries, 454 relationship, 456 state data, 454 structures and elements, 453 length distribution configuration, review details, 438 wildcard, 437 null ratio profile configuration, review results, patterns configuration, 441 delimiters, 441 profile request options, 441 review results, string data, 440 symbols, 441 tagging behavior, 441 tokenization, 441 source data, statistics configuration, review results, tolerances, 443 task connection manager, disadvantages, 429 editor's General page, package control flow, 429 profile requests, types, XML output file, 429 token-based approach, 452 value distribution configuration, distinct values, 445 results, viewer profile request, Start menu, task output, Data ProfIling task editor description, 130 General page properties, 131 Single Table Quick ProfIle Form dialog box, 132 profile requests page, Data Quality Services (DQS) connection manager, 99 DataReader destination adapter, 244 Data Transformation Services (DTS), 2, 7 Data warehouse (DW), 545 Deployment models build process project configuration, properties, environments configuration project window, creation, ETL processes, 588 parameters page and tab, 592 properties, references page, set parameter value, variables, 590 execution advanced tab, 598 connection managers tab, 597 edit literal value, execution report, 600 overview report request, 599 parameters tab, program output, property override window, script task methods, import process, 601 migration process backup option, solution page, visual studio conversion, process catalog wizard creation, CLR integration, 581 destination page, folder creation, integration services tree, results page, review page, services source page, Dimensional data ETL. See also Slowly changing dimensions (SCDs) data mart, 543 data warehouses, 543,

7 DFT data moving, 549 dead-end components, 547 fast load and bulk insert options, 550 minimize logging, 549 optimizing lookup transformations, 549 package size, 548 RunInOptimizedMode, dimension tables, 543 fact tables, 543 ODS, 545 snowflaking features, star schema features, DTS. See Data Transformation Services (DTS) Events and error handling. See also Script events event handler demonstrating package, MessageBox.ShowO method, 425 OnError, 424 tasks, 423 log provider advanced options, BIDS, detailstab, log entries, 416 OnPreExecute and OnPostExecute, 417 OnProgress, 417 SQL Server, SSIS package, stock,413 logging events data preparation tasks, 408 DFT, scripting task, 412 SQL Server tasks, workflow tasks, SSIS, Excel connection manager editor data import, SQL Server table, 240 data type conversions, 238 destination adapter configuration, 242 dialog box properties, 235 Jet drivers, 238 source editor columns page, 237 configuration, 236 spreadsheet, SSIS destination workbook, 241 project setting, 239 Excel File connection manager, 87 Execute Package task editor description, 134 package page, 134 ETL design, 134 parent-child design, 134 properties, parameter bindings page, Execute Process task editor description, 138 process page, 139 properties, Execute SQL task editor description, III expressions page, 118 General page, parameter mapping page parameter markers and names, 114 properties, 116 result set page, Extensible Markup Language CXML) source editor columns page, 504 connection manager page, 503 description, 502 task editor description, 152 General page, properties, Extract, transform, and load (ETL) bcp,2 description, 1 DTS, 2 nonfunctional requirements, 4-5 SQL Server 11, 2 tasks, 2-3 F, G File connection managers editor flat file, types of, 87 File System task editor description, 140 General page, 140 properties,

8 File Transfer Protocol (FTP) connection manager editor, task editor description, 141 file transfer page, General page, Flat file connection manager editor ADO.NET destination adapter configuration, 233 advanced page, 52 Colwnn Mappings tab, 230 colwnn-level settings, 228 colwnn settings, 209 columns page, 51 configuration, 227 editing column properties, general properties, 50, 207 Input and Output Properties tab, 231 preview page, 53 source adapter, 211 ADO.NET destination adapter connection, 232 configuration, 229 Flat files connection manager, 495 description, 495 destination editor, format dialog box, source editor, Foreach Loop container Collection page description, 197 properties, description, 196 Variable Mappings page description, properties, 201 For Loop container description, 194 For Loop page, 195 properties, 196 Heterogeneous sources and destinations, 487 Excel, 498 source editor, SQL task execution-truncate Excel worksheet, 501 flat files, 495 connection manager, 495 description, 495 destination editor, format dialog box, source editor, raw file description, 504 destination editor, 505 source editor, RDBMS, recordset destination, SQL Server, 487 command text box, 489 common table expression (CTE), 488 description, 487 OLE DB, 489, Set Query Parameters dialog box, 490 source query, 488 SQL Server Analysis Services sources, XML source editor, 502 columns page, 504 connection manager page, 503 description, 502 Hypertext Transfer Protocol (HTTP) connection manager editor description, 96 proxy page, 97 server page, 96. 1, J, K Integration services project debugging, 14 description, 14 elements, 48 Create directory, 49 Location field, 48 Name text field, 48 Solution Name field, 48 folders list, 14 Hello World, 47 DFT,57-70 elements, flat file source connection manager editor, OLE DB destination connection manager, project creation, 48 'l12

9 package annotations, package properties, 44 categories, DelayValidation, 44 ID,44 name, 44 ProtectionLevel, 44 parameters, 45 Real World, 70 control flow, 70 DFT,72-81 execute SQL task, variables, 45 Logging description, 465 enabling procedure, 466 configuration screen, data, 470 events selection, 469 log events, pop-up menu, 466 SQLlogging, SSIS log provider for SQL server, Lookup transformations advanced page, cache mode options, 299 columns page, connection page, data flow, General page, M, N Mapping, DFT EditorMappings page, 66 error, 63 error fixing, 64 package explorer, 67 Population Density data, 65 Merge Join transformation, Merge transformation, Message Queue task editor description, 170 General page description, 170 properties, Receive page description, properties, Send page description, 172 properties, 173 Microsoft Business Intelligence Development Studio (BIDS), 6 Microsoft Excel source editor, SQL task execution-truncate Excel worksheet, 501 Microsoft Message Queuing (MSMQ) connection manager, 99 Message Queue task, 170 Multicast transformation, MultifIle connection manager, 87 Multiflat File connection manager, 87 Object Linking and Embedding, Database (OLE DB) connection manager editor configuration, 213 description, 85 us. ODBC, 87 provider selection, 86 destination adapter, 214 destination editor, configuration, 222 dialog box controls, mappping page, 216 properties settings, 215 destination connection all page, configuration, 54 connection page, 55 mapping, DFT, properties pane, 55 package execution, 225 source editor, 489, 491 configuration, 220 dialog controls, ParameterO function, 221 variable mapping, 221 variables portion, 222 SSAS OLE DB editor, Operational data store (ODS),

10 . P,Q Package-level auditing AuditPackage table, 478 description, 478 EndAuditPackage, 478 StartAuditPackage, Parent-child design pattern child packages, 527 data-driven ETL child packages, collection page, column represents, execute package task, 533 expression, Foreach Loop, 534 implementation, 531 insert statements, ORDER BY clause, 534 packagetask, parameter binding, parameter mapping, result set page, variables, 533 variables mapping, dtexec.exe, 525 ETL processes, 525 logging, 531 overriding properties, 530 parameters disable package, execution task, pass values, 527 parent package, 527 script task, 526 shared configuration, 530 static package, wrapper packages, 527 Precedence constraints completion constraint, 155 configurable properties, 156 description, 155 failure constraint, 155 and parallel tasks, 157 success constraint, 155 types, 155 Rawftle description, 504 destination editor, 505 source editor, Really simple syndication (RSS), 378 Recordset destination adapter, 243 heterogeneous sources and destinations, Relational database management systems (RDBMSs), 494 Report Definition Language (RDL), 600 Report Model project, Report Server project, Report Server Project Wizard, 15 Rowset transformations aggregate advanced mode, 282 configuration, 281 data flow, operations, percentage sampling, pivot, row sampling transformation, sort comparison flags option, data flow, IsSorted properties, SortKeyPosition, unpivot, Script events component events Dts.LogO method, 421 FireErrorO method, 421 log entries, PreExecuteO and PostExecuteO methods, task events Dts.LogO method, 418 error handling, OnError event, 419 review output, 420 sample package,

11 Script task editor, 144 description, 144 properties, script page, VSTA, script editor, Scripting asynchronous transformation data flow, inbound columns, InputO_ProcessInputO method, KeywordOutput, methods, 393 outputs, PreExecuteO and PostExecuteO methods, 393 RssFeedOutput, 391 component source AddRowO method, 380 column, 377 ComponentMetaData, 380 data flow, 376 data viewer, FireProgressO method, methods, 378 pop-up menu, PostExecuteO method, PreExecuteO method, RSS feed, 378 control flow, destinations AcquireConnectionsO, 400 HTML file, input columns, 397 pop-up menu, ProcessInputRowO method, script component, 399 ScriptMain class, 399 SSIS connection, 398 SSIS package, target file, functions column maps, configuration, events, 370 Foreach file enumerator mode, 366 FireErrorO methods, 371 metadata, NET DataTable object, parameters, 370 read-only variables, script task, 367 task editor, variables, moving data, 365 OLE DB component, 365 script page, 363 string variable, 362 synchronous transformation column input and output, methods, row-numbering script, 387 SynchronousInputID, type selection, 383 variables, VSTARTScriptObjectModelBase class, Send Mail task editor mail page, properties, Sequence container, 158 Server Management Object (SMa) connection manager editor, 100 Simple Mail Transfer Protocol (SMTP) connection managers editor, Slowly changing dimensions (SCDs) nonchanging/very slowly changing dimensions, 550 type 1 dimensions data flow, DimProduct, 550 fingerprints, 554 flowchart, 551 hash code, 555 MERGE statement, 556 set-based update, updates, 552 type 2 dimensions components, SSIS, 558 result set, 556 updates, type 3 dimensions, 559 Source and destination adapters Analysis Services, 244 database, 217 ADO. NET, 226 OLE DB. See Object Linking and Embedding, Database (OLE DB) SQL Server Compact destination adapter, 226 SQL Server destination adapter, 226 DataReader destination adapter, 244 description,

12 Source and destination adapters (cant.) destination assistant description, 212 destination dialog box, 212 execution, 217 OLE DB, DFT BIDS designer, 203 components, 204 PipelineComponent class, 204 files, 226 Excel files, flat files, raw files, XML files, 243 recordset destination adapter, 243 source assistant DataPrecision properties, 210 DataScale properties, 210 description, 205 execution, 217 Flat File connection manager editor, Flat File source adapter, 211 source dialog box, 206 SQLServer Analysis Services sources, command text box, 489 common table expression (CTE), 488 Compact destination adapter, 226 description, 487 destination adapter, 226 OLE DB destination editor, source editor, 489, 491 Set Query Parameters dialog box, 490 source query, 488 SQL Server Analysis Services (SSAS), 244 SQL Server Integration Services (SSIS), 1, 43 accessing variables conditional splits, control flow designer window, derived column transformations, execute SQL, Foreach Loop containers, package explorer, parameterized queries, recordset destinations, script tasks, source types, architecture BIDS, 6 components, 7 control flow, 5 data flow, 5-6 auditing. See Auditing control flow containers, groups, data flow optimization execution tree, 518 Log Events window, data lineage. See Data lineage data types, DTS, 7 dynamic SQL, ETL-related tasks, 2-3 expressions, date operations, mathematical functions, 357 operators, string functions, features, 8 groups, Hello World. See Integration services project logging. See Logging Microsoft ETL bcp, 2 description, 1 DTS, 2 SQL Server 11, 2 nonfunctional requirements, 4-5 parameters designer window, parent-child package design, project -level, 330 types, 330 passing variables, 356 Real World. See Integration services project recommendations, 9-10 scope columns, 335 definition, dialog box, 336 functions, 334 variable options, 334 system variables containers, 339 control flow tasks, 339 event handlers, package,

13 variables default values, 337 namespaces, 337 scope, system, 325 user-defined variables, 325 SQL Server Management Studio (SSMS), 11 code snippets components, description, 39 drop and create table, description, 33 project, 37 queries, 33 SSIS queries, 42 templates code templates, 38 description, 37 parameters values, 39 tool window activity monitor, 35 bars types, 34 Cube and T -SQL queries, Object Explorer window, 34 query window, 34 toolbar buttons, SQLMOBILE connection manager editor, 101 all page, 103 connection page, 102 description, 101 properties, 103 SQL_ TruncateCensusStatistics, SSAS. See SQL Server Analysis Services (SSAS) SSIS. See SQL Server Integration Services (SSIS) Task Host container, 202 Transfer Database task editor Databases page, 176 description, 175 properties, 177 Transfer Error Messages task editor description, 177 Messages page, properties, Transfer Jobs task editor description, 180 Jobs page, properties, Transfer Master Stored Procedures task editor description, 184 properties, Stored Procedures page, Transfer SQL Server Objects task editor description, 186 Objects page, 187 properties, Union All transformation, Visual Studio tools for applications (VSTA), 364, Web Service task editor description, 149 General page, input page, Windows Management Instrumentation (WMI) connection managers editor, Data Reader task editor description, 190 Options page, properties, Event Watcher task editor description, 192 Options page, properties, X, Y, Z XML of Analysis (XMLA), 165 XML. See Extensible Markup Language (XML) 617