Monday 15 April 2013

sql - Update a flag/column value with no and set counter to create files in small chunks -


I have 60 .txt files that I'm trying to process and 1 output flat file using SSIS package Generates. Those 60.txt files are loaded in the table.

The output file will now be very large. So, I'm trying to divide this file into sections 6. So when my SSIS package runs, process 10 files and 1

produces the output file and then runs the SSIS package and generates another file. There will be a total of 6 files.

I have a SQL function that receives those specific file names in the SSIS package and then processes it. There is a column name JobDone which I am treating as a flag.

  The selection of the DISTINCT file from the table A, where JobDone is zero   

I as Table A => Bit filename , so the SSIS package JobDone lifts all the files with NULL

to take it over with the SQL statement Let's do all those 60 files. Now I want to take only 10 files at a time. I want to update a flag with nothing for example 7 With those 50 files, when my SSIS package is run, it is only 10 Will take files

Once 10 files are processed in SSIS, then updating the flag of NUL L for 10 more files from JobDone for files 10 files will run, now 40 remaining again I want to update the flag for 10 other files and want to run SSIS pacakge. Until all files are processed and up to 6 outputs are generated.

Any help would be appreciated. Thanks in advance.

If anything you ask me, my package will look something like this (before zero, / Li>

  • Img src = "https: //i.stack .imgur.com / C8TEt.png "alt =" CF ">

    Data flow will look like the following: Normally, I hate collecting a data flow but here it is understood Write the data in the file and understand the 10 filenames that you have used. Write in memory record set object (tablelist).

    data flow

    Use the Recordset object and crumble it (Post a post by Jamie Thompson for details) and this will help you draw rows from the RS object. Inside the ForEach loop, you mark those files for processing Will execute an updated statement for. < P> I have used the following script to create some data

      IF does not exist; select from sys.tables T.name = 'chains' and T.schema_id = SCHEMA_ID ( 'Dbo')) - Create source table table dbo.Chunks (sourcefile varchar (50) No NULL, JobDone bit zero, fileData varchar (500) not tap); - Add some data to COUNT SRC (select OBJECT_NAME (AC.object_id) as sourcefile, zero as bit, AC.name as fileData AC from sys.all_columns), TOP60 (Selection Top 60 SRC SRC.SourceFile command by SRC Group from SESFile, (1) (1) RC) insert into dbo.Chunks (sourcefile, JobDone, FileData) SRC.SourceFile, SRC.bit, SRC.FileData selects within TOP60 T within SRC .sourceFile = T.R.C. consists of the original file; END   

    My OLEDB source looks like the following query

      - Hold all the data related to it SELECTS.SourceFile, C.JobDone, C. FileData As far as dbo.Chunks in C.SourceFile (- Any 10 sourcefile named C that is dbo.Chunks where C.JobDone is zero is selected from the top 10. C.SourceFile has not been processed). Order by SourceFile;   

    My update statement looks like update C set as JobDone C = 1 dbo.Chunks from C.JobDone zero and C. FileName =? ;

  • No comments:

    Post a Comment