Author Topic: Suggestion needed regarding EOD data -- From Seniors and TAs  (Read 3860 times)

0 Members and 1 Guest are viewing this topic.

Offline ksestocks_dot_com

  • Research
  • New Member
  • *****
  • Posts: 31
  • Thanks Received: 3
    • View Profile
Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #-1 on: March 14, 2013, 04:51:15 PM »
Previously when KSE EOD feed was available at their website, I used it to prepare EOD data for ksestocks and patched it with the closing rates obtained from the daily summary which is still present on their site.

Now the problem is that since Oct 2012, KSE is not providing the daily feed due to which I had to fall back on other sources (Moosani and SCS Trade being the most prominent). I knew their could be some issues in the reliability of data but the companies in which the data corruption was present were mostly very low volume non-frequently traded companies therefore I ignored it for time being due to some of my own commitments. Also note that the data corruption which I am talking about, and which will be explained below in more detail, was only present on days where the said company opened higher than previous close and did not go below the last close during the session. Similarly if it opened lower than last close and did not go higher than last close.

Now, when I have some time, I was analyzing the EOD data obtained from three sources (KSE, Moosani and SCS Trade) to find out which source or a combination of sources to use in future. To my surprise, both Moosani and SCS were using last close as the open price for companies such as mentioned above whereas actual open for all other high volume companies. By doing this, they also had to forge the high/low accordingly. for example on 13th Mar 2013

KSE reports >> CENI,13,12.75,12.72,12.75,47000
Moosani >> CENI,13,13,12.72,12.75,47000
SCS >> CENI,13.00,13.00,12.72,12.75,47000

Now one thing I am sure about is that KSE is reporting the actual high which CENI has hit during the session. Moosani and SCS used last close as open therefore they forged the high. Other discrepancies for 13th Mar 2013 are as follows;

CENI: KSE differs Moosani on >> high
IGIIL: KSE differs Moosani on >> low
DKTM: KSE differs Moosani on >> high
RCML: KSE differs Moosani on >> high, low
SSML: KSE differs Moosani on >> low
YOUW: KSE differs Moosani on >> high
MOON: KSE differs Moosani on >> high, low
HUSS: KSE differs Moosani on >> high, low, close
MZSM: KSE differs Moosani on >> low
KOHE: KSE differs Moosani on >> low
BPL: KSE differs Moosani on >> high
INDU: KSE differs Moosani on >> high
SAPL: KSE differs Moosani on >> high, low
LINDE: KSE differs Moosani on >> low
MFFL: KSE differs Moosani on >> low
DREL: KSE differs Moosani on >> high, low, close
AKDCL: KSE differs Moosani on >> high

I could not upload the related data files here. If you need those please send a pm and I can email.

This list also includes 2 companies which were not traded at all and therefore can be ignored but still there are 15 left.

Now my question is which pattern we shall follow. the options which I can think about are
1. Take the High, Low, Close, Volume from the KSE data and use Open from either moosani or SCS trade (adjust open to either high or low where there is an error in Moosani)
2. Use Open,high,low from Moosani/SCS and use close & volume from KSE (and forget about the error)

If anyone has any other option/suggestion please do share.

Pakinvestorsguide

Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #-1 on: March 14, 2013, 04:51:15 PM »

Offline Farzooq

  • Administrator
  • Senior Member
  • *****
  • Posts: 19978
  • Thanks Received: 196
    • View Profile
Data with minor discrepancies is acceptable. Please continue with what is convenient.

Regards
TOP PICKS
Engro fatima ogdc pol pso dgkc mlcf kapco npl ubl atrl nml efoods aicl hcar searl

Offline Ayub

  • Junior Member
  • **
  • Posts: 232
  • Thanks Received: 13
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #1 on: March 14, 2013, 06:19:37 PM »
Previously when KSE EOD feed was available at their website, I used it to prepare EOD data for ksestocks and patched it with the closing rates obtained from the daily summary which is still present on their site.

Now the problem is that since Oct 2012, KSE is not providing the daily feed due to which I had to fall back on other sources (Moosani and SCS Trade being the most prominent). I knew their could be some issues in the reliability of data but the companies in which the data corruption was present were mostly very low volume non-frequently traded companies therefore I ignored it for time being due to some of my own commitments. Also note that the data corruption which I am talking about, and which will be explained below in more detail, was only present on days where the said company opened higher than previous close and did not go below the last close during the session. Similarly if it opened lower than last close and did not go higher than last close.

Now, when I have some time, I was analyzing the EOD data obtained from three sources (KSE, Moosani and SCS Trade) to find out which source or a combination of sources to use in future. To my surprise, both Moosani and SCS were using last close as the open price for companies such as mentioned above whereas actual open for all other high volume companies. By doing this, they also had to forge the high/low accordingly. for example on 13th Mar 2013

KSE reports >> CENI,13,12.75,12.72,12.75,47000
Moosani >> CENI,13,13,12.72,12.75,47000
SCS >> CENI,13.00,13.00,12.72,12.75,47000

Now one thing I am sure about is that KSE is reporting the actual high which CENI has hit during the session. Moosani and SCS used last close as open therefore they forged the high. Other discrepancies for 13th Mar 2013 are as follows;

CENI: KSE differs Moosani on >> high
IGIIL: KSE differs Moosani on >> low
DKTM: KSE differs Moosani on >> high
RCML: KSE differs Moosani on >> high, low
SSML: KSE differs Moosani on >> low
YOUW: KSE differs Moosani on >> high
MOON: KSE differs Moosani on >> high, low
HUSS: KSE differs Moosani on >> high, low, close
MZSM: KSE differs Moosani on >> low
KOHE: KSE differs Moosani on >> low
BPL: KSE differs Moosani on >> high
INDU: KSE differs Moosani on >> high
SAPL: KSE differs Moosani on >> high, low
LINDE: KSE differs Moosani on >> low
MFFL: KSE differs Moosani on >> low
DREL: KSE differs Moosani on >> high, low, close
AKDCL: KSE differs Moosani on >> high

I could not upload the related data files here. If you need those please send a pm and I can email.

This list also includes 2 companies which were not traded at all and therefore can be ignored but still there are 15 left.

Now my question is which pattern we shall follow. the options which I can think about are
1. Take the High, Low, Close, Volume from the KSE data and use Open from either moosani or SCS trade (adjust open to either high or low where there is an error in Moosani)
2. Use Open,high,low from Moosani/SCS and use close & volume from KSE (and forget about the error)

If anyone has any other option/suggestion please do share.

Dear Brother AA,

1. Welcome. Posting after a long time? I pray all is well.

2. Option 1. "1. Take the High, Low, Close, Volume from the KSE data and use Open from either moosani or SCS trade (adjust open to either high or low where there is an error in Moosani)" is recommended please.

3. Thanks for your perpetual endeavours & help.

4. May Allah reward you.

Regards,
QURAN 2:32. GLORY BE TO ALLAH, WE HAVE NO KNOWLEDGE EXCEPT WHAT YOU HAVE TAUGHT US. VERILY, IT IS YOU, THE ALL-KNOWER, THE ALL-WISE.

Offline ksestocks_dot_com

  • Research
  • New Member
  • *****
  • Posts: 31
  • Thanks Received: 3
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #2 on: March 14, 2013, 07:40:20 PM »
@ Farzooq: well for me right now both options are equally convenient/difficult because both require at-least 2 sources so I have to decide between the two options according to the wishes of technical analysts who use the data.

@Ayub: Personally I am also in the favour of option 1 but lets wait to see what other researchers have to say.

Offline M&M

  • KSE100 Trader
  • Administrator
  • Senior Member
  • *****
  • Posts: 4428
  • Thanks Received: 109
  • Who doesn't like chocolates and profits?
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #3 on: March 14, 2013, 08:16:07 PM »
@ksestock - option 1 is good because kse data has everything correct except the opening price
go with scstrade opening price and take the rest from kse official site.

have you ever encountered any errors in scstrade 'Open' price for stocks ?

thank you for providing the EOD data  :thanks:
"The only true wisdom is in knowing you know nothing." - Socrates

Guidelines | Messages | Follow me

Offline ksestocks_dot_com

  • Research
  • New Member
  • *****
  • Posts: 31
  • Thanks Received: 3
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #4 on: March 14, 2013, 11:23:08 PM »
@M&M: Yes SCS and Mossani data is 99% identical. In the example I quoted above SCS was also using last close and sometimes I feel that SCS (it shouldn't be like this because they are broker themselves and have similar access to KATs) is also using the Moosani data because when ever there is a delay on moosani, scs is also delayed. Remaining 1% diiference is because SCS misses data for some symbols constantly e.g DBCI, SARC, REDCO and few others

Offline M&M

  • KSE100 Trader
  • Administrator
  • Senior Member
  • *****
  • Posts: 4428
  • Thanks Received: 109
  • Who doesn't like chocolates and profits?
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #5 on: March 15, 2013, 10:45:10 AM »
@M&M: Yes SCS and Mossani data is 99% identical. In the example I quoted above SCS was also using last close and sometimes I feel that SCS (it shouldn't be like this because they are broker themselves and have similar access to KATs) is also using the Moosani data because when ever there is a delay on moosani, scs is also delayed. Remaining 1% diiference is because SCS misses data for some symbols constantly e.g DBCI, SARC, REDCO and few others

hmm
sometimes 100 index 'high' 'low' 'volume' also mismatch that from kse official data .. for ex.
(3/12/2013 100 index volume mismatch)
100 index 'open' is also something that differs even among security firm's TA data.

I think the best option is to take data from kse offical and replace 'Open' from scstrade/moosani
"The only true wisdom is in knowing you know nothing." - Socrates

Guidelines | Messages | Follow me

Offline ksestocks_dot_com

  • Research
  • New Member
  • *****
  • Posts: 31
  • Thanks Received: 3
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #6 on: March 15, 2013, 01:08:37 PM »
Ok, so the consensus is apparently on option 1. Now the pattern will be as follows from today InshAllah

Take Open, High, Low from KSE Feed
Take Close, Volume from KSE summary

If process fails due to non availability of KSE Feed, fallback to alternate

Take all symbols present in KSE summary (Including Futures, TFCs, PSs)
Take Open from Moosani if the symbol is present Else take Open from SCS if symbol present Else take Open from KSE summary
Take High,Low,Close,Volume from KSE summary
If the Open is higher than High then adjust it equal to High
If the Open is lower than Low then adjust it equal to Low

Offline Ayub

  • Junior Member
  • **
  • Posts: 232
  • Thanks Received: 13
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #7 on: March 18, 2013, 12:26:38 AM »
Ok, so the consensus is apparently on option 1. Now the pattern will be as follows from today InshAllah

Take Open, High, Low from KSE Feed
Take Close, Volume from KSE summary

If process fails due to non availability of KSE Feed, fallback to alternate

Take all symbols present in KSE summary (Including Futures, TFCs, PSs)
Take Open from Moosani if the symbol is present Else take Open from SCS if symbol present Else take Open from KSE summary
Take High,Low,Close,Volume from KSE summary
If the Open is higher than High then adjust it equal to High
If the Open is lower than Low then adjust it equal to Low

Dear Brother,

Please see, if it is useful to check logic in the last two lines differently for White & Black Candles as under:
 
In case Close is higher than Open (i.e. White candle), if the Open is lower than Low then adjust it equal to Low

In case Close is lower than Open (i.e. Black Candle), if the Open is higher than High then adjust it equal to High

Regards,

QURAN 2:32. GLORY BE TO ALLAH, WE HAVE NO KNOWLEDGE EXCEPT WHAT YOU HAVE TAUGHT US. VERILY, IT IS YOU, THE ALL-KNOWER, THE ALL-WISE.

Offline ksestocks_dot_com

  • Research
  • New Member
  • *****
  • Posts: 31
  • Thanks Received: 3
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #8 on: March 18, 2013, 06:03:25 PM »
Ok, so the consensus is apparently on option 1. Now the pattern will be as follows from today InshAllah

Take Open, High, Low from KSE Feed
Take Close, Volume from KSE summary

If process fails due to non availability of KSE Feed, fallback to alternate

Take all symbols present in KSE summary (Including Futures, TFCs, PSs)
Take Open from Moosani if the symbol is present Else take Open from SCS if symbol present Else take Open from KSE summary
Take High,Low,Close,Volume from KSE summary
If the Open is higher than High then adjust it equal to High
If the Open is lower than Low then adjust it equal to Low

Dear Brother,

Please see, if it is useful to check logic in the last two lines differently for White & Black Candles as under:
 
In case Close is higher than Open (i.e. White candle), if the Open is lower than Low then adjust it equal to Low

In case Close is lower than Open (i.e. Black Candle), if the Open is higher than High then adjust it equal to High

Regards,


In cases where we have the actual open available (95% cases) we use the actual open the last two lines refer to the remaining 5% cases.

Open higher than High can only occur on a black candle day. Reason: KSE uses last day's close as next day's open. Now if a scrip could not touch the last close then it should be a negative open. Now we have three possibilities in this case
  • It opened at a price and went down and then never crossed the open price during the session making the open as the highest price
  • It opened somewhere between the high and the low.
  • It opened at the lowest price and went up and never crossed below the open during the session making open as the lowest price
In the above three scenarios we can adjust the open price either to the low or the high because we have no means to discover the open price if it was case 2. So this is the compromise we have to make for such scrips. These are mostly the Futures or very low traded companies.

But again this is not my preference instead it is the only option we have in a scenario where KSE is not willing to publicise the daily feed and the other available sources are using last day's close as next day's open.

Offline Ayub

  • Junior Member
  • **
  • Posts: 232
  • Thanks Received: 13
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #9 on: March 19, 2013, 12:25:32 PM »
Ok, so the consensus is apparently on option 1. Now the pattern will be as follows from today InshAllah

Take Open, High, Low from KSE Feed
Take Close, Volume from KSE summary

If process fails due to non availability of KSE Feed, fallback to alternate

Take all symbols present in KSE summary (Including Futures, TFCs, PSs)
Take Open from Moosani if the symbol is present Else take Open from SCS if symbol present Else take Open from KSE summary
Take High,Low,Close,Volume from KSE summary
If the Open is higher than High then adjust it equal to High
If the Open is lower than Low then adjust it equal to Low

Dear Brother,

Please see, if it is useful to check logic in the last two lines differently for White & Black Candles as under:
 
In case Close is higher than Open (i.e. White candle), if the Open is lower than Low then adjust it equal to Low

In case Close is lower than Open (i.e. Black Candle), if the Open is higher than High then adjust it equal to High

Regards,


In cases where we have the actual open available (95% cases) we use the actual open the last two lines refer to the remaining 5% cases.

Open higher than High can only occur on a black candle day. Reason: KSE uses last day's close as next day's open. Now if a scrip could not touch the last close then it should be a negative open. Now we have three possibilities in this case
  • It opened at a price and went down and then never crossed the open price during the session making the open as the highest price
  • It opened somewhere between the high and the low.
  • It opened at the lowest price and went up and never crossed below the open during the session making open as the lowest price
In the above three scenarios we can adjust the open price either to the low or the high because we have no means to discover the open price if it was case 2. So this is the compromise we have to make for such scrips. These are mostly the Futures or very low traded companies.

But again this is not my preference instead it is the only option we have in a scenario where KSE is not willing to publicise the daily feed and the other available sources are using last day's close as next day's open.

Dear Brother AA,

1. Thanks for a detailed reply.

2. I agree to your logic.

3. Please accept my thanks for your untiring efforts to solve this quagmire.

4. I admire your deep understanding of the KSE EOD Pandora's box.

5. In reply to  your above "it is the only option we have in a scenario where KSE is not willing to publicise the daily feed", I can only offer consolation:

"Taqdeer Ke Qazi Ka Ye Fatwa Hai Azal Se
Hai Jurm-e-Zaeefi Ki Saza Marg-e-Mafajat!"

Regards, 
« Last Edit: March 19, 2013, 09:41:30 PM by Ayub »
QURAN 2:32. GLORY BE TO ALLAH, WE HAVE NO KNOWLEDGE EXCEPT WHAT YOU HAVE TAUGHT US. VERILY, IT IS YOU, THE ALL-KNOWER, THE ALL-WISE.

Offline M&M

  • KSE100 Trader
  • Administrator
  • Senior Member
  • *****
  • Posts: 4428
  • Thanks Received: 109
  • Who doesn't like chocolates and profits?
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #10 on: March 20, 2013, 08:00:57 PM »
today's high low data are incorrect for KSE-100
"The only true wisdom is in knowing you know nothing." - Socrates

Guidelines | Messages | Follow me

Offline ksestocks_dot_com

  • Research
  • New Member
  • *****
  • Posts: 31
  • Thanks Received: 3
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #11 on: March 21, 2013, 02:09:05 AM »
today's high low data are incorrect for KSE-100
Sometimes I feel that KSE guys deliberately do this to me  huhu
The index data comes from the daily index fluctuations file and if it contains wrong data then GOD help me.

Offline M&M

  • KSE100 Trader
  • Administrator
  • Senior Member
  • *****
  • Posts: 4428
  • Thanks Received: 109
  • Who doesn't like chocolates and profits?
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #12 on: March 21, 2013, 10:34:28 AM »
today's high low data are incorrect for KSE-100
Sometimes I feel that KSE guys deliberately do this to me  huhu
The index data comes from the daily index fluctuations file and if it contains wrong data then GOD help me.
:console:

how about taking the index data from scstrade ?
"The only true wisdom is in knowing you know nothing." - Socrates

Guidelines | Messages | Follow me

Offline ksestocks_dot_com

  • Research
  • New Member
  • *****
  • Posts: 31
  • Thanks Received: 3
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #13 on: March 21, 2013, 07:23:46 PM »
today's high low data are incorrect for KSE-100
Sometimes I feel that KSE guys deliberately do this to me  huhu
The index data comes from the daily index fluctuations file and if it contains wrong data then GOD help me.
:console:

how about taking the index data from scstrade ?
I don't want to taunt about other sites cause they might have there own problems but the fact is that they aren't any better either. Check today's index lows on their daily activity page. Those are all set to yesterday's close.

Our main problem is at source. The kse_index.lis which is present in daily downloads is a 14-15 seconds tick file. On the KSE Trade Screen page they fetch data from some other similar source (i.e. 14-15 second ticks) which is not publicized and both the sources are not synchronised as well hence the difference arises. In dull market days it is in fraction/decimals and in volatile days it is between 1-5 points

Offline M&M

  • KSE100 Trader
  • Administrator
  • Senior Member
  • *****
  • Posts: 4428
  • Thanks Received: 109
  • Who doesn't like chocolates and profits?
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #14 on: March 21, 2013, 10:14:00 PM »
today's high low data are incorrect for KSE-100
Sometimes I feel that KSE guys deliberately do this to me  huhu
The index data comes from the daily index fluctuations file and if it contains wrong data then GOD help me.
:console:

how about taking the index data from scstrade ?
I don't want to taunt about other sites cause they might have there own problems but the fact is that they aren't any better either. Check today's index lows on their daily activity page. Those are all set to yesterday's close.

Our main problem is at source. The kse_index.lis which is present in daily downloads is a 14-15 seconds tick file. On the KSE Trade Screen page they fetch data from some other similar source (i.e. 14-15 second ticks) which is not publicized and both the sources are not synchronised as well hence the difference arises. In dull market days it is in fraction/decimals and in volatile days it is between 1-5 points

yeah that's right.
chalo never mind, little mismatches can be manually corrected by users.
"The only true wisdom is in knowing you know nothing." - Socrates

Guidelines | Messages | Follow me

Offline Ayub

  • Junior Member
  • **
  • Posts: 232
  • Thanks Received: 13
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #15 on: March 22, 2013, 10:57:01 AM »
The crux of the matter is that if you’re testing on only the CLOSE data, you’re on solid ground.

If you’re using HIGH and LOW on EOD, you must be careful.
QURAN 2:32. GLORY BE TO ALLAH, WE HAVE NO KNOWLEDGE EXCEPT WHAT YOU HAVE TAUGHT US. VERILY, IT IS YOU, THE ALL-KNOWER, THE ALL-WISE.

Offline M&M

  • KSE100 Trader
  • Administrator
  • Senior Member
  • *****
  • Posts: 4428
  • Thanks Received: 109
  • Who doesn't like chocolates and profits?
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #16 on: April 04, 2013, 07:07:29 PM »
i think moosani is the one who takes data from scstrade and not the other way round
"The only true wisdom is in knowing you know nothing." - Socrates

Guidelines | Messages | Follow me

Offline SimonKatich

  • New Member
  • *
  • Posts: 16
  • Thanks Received: 0
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #17 on: December 21, 2015, 06:52:03 AM »
Aisi information boht hi useful hoti ha kisi bhe trader k lia. Mai stock trading boht hi kam karta hon kiu k mujhe Forex zada pasand ha aur zada profit ka chance bhe is mai hi hota ha. Mai OctaFX broker k sath work kar rah hon jo trading karna boht hi asan kardeti ha khas tor pay in k low spread of 0.2 pips, high leverage up to 1.500 aur sath mai 8 dollars no deposit bonus bhe available ha jis se trading karna kafi asan hojata ha.

Offline Ahmedkhan

  • Junior Member
  • **
  • Posts: 117
  • Thanks Received: 0
    • View Profile
Re: Suggestion needed regarding EOD data -- From Seniors and TAs
« Reply #18 on: July 27, 2016, 04:31:18 AM »
Sahi kaha apne Katich, agar hume successful ban na ha to aisi information boht hi zada zaroori ha lena aur sirf lena hi nahi balke samajhna agar hum aise karein gay to hum aram se success hasil kar sakte hain magar hum agar ise value nahi degay to success hasil karna boht muskil hoga. Mai bhe OctaFX broker k sath kam kar rah hon aur ya waqai mai boht achi company ha khas tor pay in ka educational setup jo boht hi ala ha.

Tags: