Beta testers wanted for new CDR report tool (UPDATED)

UPDATE: 2011-08-22, beta2 version released.
UPDATE: 2011-08-11, beta version released.
I have found a new nice report program for Asterisk CDR. It is written by Anthony Joseph Messina, modified by Igor Okunev and it is called Asterisk CDR Viewer.
I rewrote the code into a FreePBX Module. The code is in alpha stage now, there might be some quirks in it that I have not thought about. My tests has been on a rather large CDR database that contains records dated from 2006 up to today with no strange effects.
Download the BETA FreePBX CDR Viewer Upload it via Module Admin.
You will find it in Tools section with the title CDR Viewer.
It will work with FreePBX 2.7 and up.
Note that ticking Concurrent calls and run a report will take some (considerable) time.
Give it some testing and let me know what you think.
What I like to know is Can this module replace the current report tool that FreePBX use today?
In this release only .wav files are supported, that might change in later releases.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)



It looks pretty neat. Did
It looks pretty neat. Did not appear to break anything on a FreePBX ISO install. I am not sure I am seeing a way to search on DID dialed as to track inbound calls. Then again that might require some other manipulation.
Robert Keller - VoIPologist
FreePBX Training Opportunities - Click Here
Get Official Paid Support - Click Here
Can you add grouping of
Can you add grouping of numbers in it (Let's say two companies have one FreePBX server, but the CDR has to be made for two seperate group of numbers)?
working great
Nice addition - I see on the homepage that you can add links to the recordings. The instructions on their website relates to an installation done outside of freepbx. How would I edit (or where would I find it rather) the config.inc.php file that relates to this when installed as a freepbx module?
And has anybody tried linking the sound files already?
Thanks for any input
cosmicwombat: DID will (most
cosmicwombat: DID will (most likely) be added to 2.10, but that requires a configuration change switching from cdr_mysql for CDR to res_odbc/cdr_adaptive_odbc.conf. By doing that we can add whatever field we want to CDR. This module will be updated when more testing have been done.
black187: Can you elaborate the question?
mlungu: All settings in config.inc.php are now read from FreePBX using the settings that already exists. However, a bug prevents that (my bad).
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
Done - but how do I see the call recordings?
Hi Mikael,
Uploaded the new one - do you have any idea how I would now see the link to the call recording??
Today I finally had the time
Today I finally had the time to really test on a live environment, there are a few more bugs to sort out. And that is why the module is still in alpha stage:-)
Give me a couple of days to test the link to recordings function and add code so that one can play or download it.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
ok ok ok
No problem :-) I know it's still alpha. I just get excited a bit easily :-). Your modules are highly appreciated!
Regarding grouping of number
@mickecarlsson: The idea is - when you click on the CDR viewer, you can choose or make a group, which contains some extensions. And you can then view the CDR records only for the selected/made group.
@black187, oh, I see. Hmmm,
@black187, oh, I see. Hmmm, let me think about that. No promises made :-)
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
multiple asterisk-regexp for src/dst fields
I made small changes to support multiple regular expressions separated by a comma in the fields SRC/DST.
For example:
* src = '_2XXN, _562., _.0075' - src any of these numbers
* src = '_!2XXN, _562., _.0075' - src any other than these numbers
If it is normal for you, you can add this changes to the module.
Index: include/config.inc.php
===================================================================
--- include/config.inc.php (revision 50)
+++ include/config.inc.php (working copy)
@@ -170,7 +170,7 @@
$number = substr($number,1);
} elseif ( '_' == substr($number,0,1) ) {
$number_chars = preg_split('//', substr($number,1), -1, PREG_SPLIT_NO_EMPTY);
- $number = '^';
+ $number = '';
foreach ($number_chars as $chr) {
if ( $chr == 'X' ) {
$number .= '[0-9]';
@@ -187,7 +187,6 @@
}
}
$_POST[ $source_data .'_mod' ] = 'asterisk-regexp';
- $number .= '$';
}
return $number;
}
Index: index.php
===================================================================
--- index.php (revision 50)
+++ index.php (working copy)
@@ -116,7 +116,22 @@
}
break;
case "asterisk-regexp":
- $$key = "AND $key $pre_like RLIKE '$val[0]'";
+ $ast_dids = preg_split('/\s*,\s*/', $val[0], -1, PREG_SPLIT_NO_EMPTY);
+ $ast_key = '';
+ foreach ($ast_dids as $did) {
+ if (strlen($ast_key) > 0 ) {
+ if ( $pre_like == ' NOT ' ) {
+ $ast_key .= " and ";
+ } else {
+ $ast_key .= " or ";
+ }
+ if ( '_' == substr($did,0,1) ) {
+ $did = substr($did,1);
+ }
+ }
+ $ast_key .= " $key $pre_like RLIKE '^$did\$'";
+ }
+ $$key = "AND ( $ast_key )";
break;
case "begins_with":
default:
Community, please welcome
Community, please welcome Igor Okunev, the author behind Asterisk CDR Viewer.
@igor_okunev, I have updated FreePBX svn with the above changes.
For those who wants to test out the new feature, download page.cdr.php and functions.inc.php and replace those in modules/cdr.
Run amportal chown after the files have been replace to set correct owner for the files.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
The CDR Viewer is now
The CDR Viewer is now released as beta.
Download the BETA FreePBX CDR Viewer Upload it via Module Admin.
You will find it in Tools section with the title CDR Viewer.
In this release you can play recorded calls. NOTE: Only .wav is supported at the moment.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
New version seemed to
New version seemed to install fine over earlier alpha5 version on FreePBX 2.9 system (FreePBX distro) but on a FreePBX 2.8/Elastix system, when the orange box came up the frog never appeared, nor did any text at all - it just froze at that point (I left it alone for several minutes) and it stayed that way until I closed that page and opened another. I then completely uninstalled the alpha5 version, and tried to install the new version with the same result. All indications are that the install never actually begins.
I re-uploaded alpha5 on that system and it installed without a hitch. Is the new version perhaps limited to FreePBX 2.9 and above?
No, but there are code in it
No, but there are code in it that is PHP 5 only. Hmm, I need to check that before installation.
Could you check the php used in that system that failed? Do it with php -v in a shell?
Thanks for the report, I will try to install it on an Elastix system and see what happens.
Update: Now that I think of it, it must be the update.php scrip that fails, it tries to add a field in the asteriskcdrdb/cdr database/table.
I definitely needs to test that.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
# php -v PHP 5.2.17 (cli)
# php -v
PHP 5.2.17 (cli) (built: Aug 9 2011 20:12:01)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2010 Zend Technologies
Obviously that's been updated since the original install (yesterday, in fact, because of the security issue).
I just checked and it did not succeed in adding the field. I wonder if there's a user/password issue, perhaps?
OK, that pointed me to the
OK, that pointed me to the solution, could you edit install.php from this:
to this:
I forgot that we always include that from 2.9 and onwards.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
This time the orange box
This time the orange box came up and it said "Please wait while module actions are performed" and that was it… no frog or other text. It just sits there. Again I totally uninstalled the previous version (even deleted the cdr directory after doing that), uploaded the new version, and made the change you requested.
I decided to try an experiment and added a few debug lines:
// Update cdr database with did field
//
out("debug 1");
global $db;
out("debug 2");
$sql = "SELECT did FROM asteriskcdrdb.cdr";
out("debug 3");
$confs = $db->getRow($sql, DB_FETCHMODE_ASSOC);
out("debug 4");
if (DB::IsError($confs)) { // no error... Already there
NOTHING appeared in the orange box. So for some reason, it sure looks like the install script isn't being executed at all. Yet as I say, the older alpha version installs fine (but there is no install.php in that one, it appears)
OK, I will look at it later
OK, I will look at it later tonight and install it on an old 2.8 setup. Thanks for the feedback.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
You're welcome. I found the
You're welcome. I found the answer, see my previous post (which will probably appear BELOW this one because I forgot to attach it as a reply, and apparently there's no way to delete a message if you did it wrong).
GOT IT! You were on the
GOT IT!
You were on the right track, you just needed to swipe a little more code from a similar install.php. THIS worked:
// Update cdr database with did field // global $db; global $amp_conf; if (! function_exists("out")) { function out($text) { echo $text."<br />"; } } $sql = "SELECT did FROM asteriskcdrdb.cdr"; $confs = $db->getRow($sql, DB_FETCHMODE_ASSOC);Note that JUST adding "global $amp_conf;" did NOT do the trick, so the "out" function thing must have been the key.
I spotted that but ran out
I spotted that but ran out of time before heading out to work. The $amp_conf is not needed as we don't have any calls to retrieve amp_conf variables.
I have republished the module with all fixes, please download it again and test.
Thanks for the feedback.
Have you tested the playback of recordings?
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
The revised one installs
The revised one installs fine.
Haven't tested playback of recordings because I've never actually had a reason to create a recording (I assume you mean a recoding of a call, and I don't even remember the code for that offhand), therefore there are none on the system, and I need to leave in a few minutes. Maybe later tonight I will give that a try.
Thanks for the feedback. As
Thanks for the feedback. As for recording, just set an extension to Record Incoming to Always and dial into it a couple of times. Then set it back to what it was.
Any other takers for testing the playback of recorded calls?
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
Okay, I tested playback of
Okay, I tested playback of recordings on a Mac (OS X 10.5.8) using Firefox, Safari, Chrome, and Opera. In Firefox, Safari and Chrome they played fine using the QuickTime plug-in. In Opera they played fine also although some of the controls were not available, for example you couldn't pause the recording. I think this may be an Opera or a QuickTime issue.
Also tested on Firefox, Chrome and Opera on Ubuntu Linux 10.10. In Firefox it plays fine there also, using the "Totem Browser Plug-In". In Chrome and Opera, it just says "Missing Plug-in" where the control should be. There's probably a fix for that, but since I don't use that system for web browsing I'm not going to worry about it, and I'm inclined to think it's not a problem with your software because a quick Google search tells me that a LOT of people have problems getting sound to play in Ubuntu, when using browsers other than Firefox (and Google Chrome users apparently even have issues getting sound to play under some versions of Windows, so there's that).
Just a note for anyone
Just a note for anyone testing this that wants to see the DID field actually populated. For now you can add a context at the bottom of /etc/asterisk/extensions_custom.conf
[custom-from-trunk-did]exten => _X!,1,Set(CDR(did)=${EXTEN})
exten => _X!,n,Goto(from-trunk,${EXTEN},1)
exten => h,1,Macro(hangupcall,)
Then in any trunk that handles incoming calls, set the context statement to context=custom-from-trunk-did (assuming the statement was originally context=from-trunk).
If the context statement in a trunk points to something other than from-trunk, there are a couple of ways to handle it. If it went to another custom context that you created, just add a line to that custom context that includes the Set(CDR(did)=${EXTEN}) instruction. If you went to some other valid internal context (for example, from-pstn-e164-us, from-pstn-toheader, etc.) just create another corresponding context in extensions_custom conf, for example:
[custom-from-pstn-e164-us-did]exten => _X!,1,Set(CDR(did)=${EXTEN})
exten => _X!,n,Goto(from-pstn-e164-us,${EXTEN},1)
exten => h,1,Macro(hangupcall,)
And then change the trunk context line to point to the appropriate custom context. But keep in mind that whenever this module becomes part of the official distribution, code to set the DID field in the CDR records will probably be added to from-trunk and similar contexts, making these custom contexts redundant at best.
Testing on PIAF
Ran into an issue testing the CDR Viewer on my PBX in a Flash setup FreePBX 2.9. If I raise the result limit to 10,000 or higher and check the export to CSV it crashes (after about 10 seconds). However if I try to do the 10,000 and only check the cdr search it works.
Well, I can put in a limit
Well, I can put in a limit on that, 10,000 entries are a lot of entries:-)
I guess that memory is exhausted in php and that is why it crashes. Is there any error in apache error_log?
Update:
PHP Fatal error: Maximum execution time of 30 seconds exceeded in /var/www/html/admin/modules/cdr/crypt.php on line 0, referer: http://192.168.0.205/admin/config.php?type=tool&display=cdr
Could you test this, in functions.inc.php add the following from:
function cdr_export_csv($csvdata) { global $db; $fname = "cdr__" . (string) time() . $_SERVER["SERVER_NAME"] . ".csv";to
function cdr_export_csv($csvdata) { global $db; set_time_limit(3000); $fname = "cdr__" . (string) time() . $_SERVER["SERVER_NAME"] . ".csv";Then test it again.
Update 2:
Just tested it with 10,000 records, it took almost a minute, then I could download it.
Thanks for the report.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
Thanks It works now without
Thanks It works now without crashing. However took about 2 min on the system I am running. Bad thing is that I need to create a csv of our monthly call logs for reporting. So I went directly to the database and queried the past month August 1st through August 31 I have 62,000 + lines!!!! So any suggestions on a better way. I was testing making a cron job to query the DB and then upload it to my FTP but kept getting issues with too many arguments. Its been a long week so maybe I am missing something.. Thanks for the help in advance
Unfortunately, it will take
Unfortunately, it will take some time to process 62,000+ lines.
I wonder how long it took for the old cdr report tool to process that. Hmmm, I need to test it.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
The old reports take a long
The old reports take a long time too. Taking 2+ minutes would not be unreasonable.
BF
Bill/W5WAF
IT Specialist
City of Vicksburg, MS
I have a CDR DB with 7M+
I have a CDR DB with 7M+ records. Would you like me to see what happens with that data set?
That was 2+ minutes on the
That was 2+ minutes on the 10,000. The issue is that some months we may have 60,000 lines and others we may have 70,000 or even 80,000 lines. When I query the Database manually and email to my self it creates a ~13Meg file of all 60,000+ lines, in about 1 min and then I have it in my Personal Inbox (Not PBX Mailbox) within another minute or 2.
I am looking into it, it is
I am looking into it, it is the foreach loop that is taking time.
If I select 10,000 records, the select for that take about 0,4 seconds, the foreach loop that populates the csv take 43 seconds.
I will try to see if I can optimize the code foreach or do another approach.
@SkykingOH, I have a rather large cdr database to play with:
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
OK, I have now tweaked the
OK, I have now tweaked the code a bit, I found where the time was spent in the code.
With 10,000 records, it took about 50 seconds to generate the csv export, with this code it will take about 0,6 seconds to export to csv.
In functions.inc.php replace the function cdr_export_csv with this code:
function cdr_export_csv($csvdata) { set_time_limit(3000); $fname = "cdr__" . (string) time() . $_SERVER["SERVER_NAME"] . ".csv"; $csv_header ="calldate,clid,src,dst,dcontext,channel,dstchannel,lastapp,lastdata,duration,billsec,disposition,amaflags,accountcode,uniqueid,userfield\n"; $data = $csv_header; foreach ($csvdata as $csv => $value) { $tmp = ''; foreach ($value as $csv_key => $csv_val) { // If the string contains a comma, enclose it in double-quotes. if (strpos($csv_val, ",") !== FALSE) { $csv_val = "\"" . $csv_val . "\""; } $tmp .= ",$csv_val"; } $data .= substr($tmp, 1)."\n"; } cdr_download($data, $fname); return; }Please test and report back.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
On my system it seems to
On my system it seems to break it like prior to adding the time limit.
Log show this error
Call to undefined function cdr_ob_clean_all() in /var/www/html/admin/modules/cdr/functions.inc.php on line 160,
Update:
My mistake went a bit too far on the function selection!!
You probably revoed the
You probably removed the function when pasting in the fix. That function code was after after the replacement code.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
Works great got 62,000 +
Works great got 62,000 + results in <30. I will keep testing. Any Possibility of adding a find all results to this?
Something is causing a
Something is causing a column shift with Lastapp and Lastdata Duration, Billable sec and proceeding columns are off. Looking into it.
Update:
lastapp column is shifting on us. Also clid is flowing over into src on rows with long clid's. Resulting in additional column shifts on certain rows.
OK, I did some more testing
OK, I did some more testing and found where php suffers when allocation data.
This is time consuming in php:
Execution time: 43.395808935165 seconds
By replacing that with:
Execution time: 0.1939640045166 seconds
The speedup in php is enormous.
So, restore the functions.inc.php, then change according to this:
Index: functions.inc.php =================================================================== --- functions.inc.php (revision 12476) +++ functions.inc.php (working copy) @@ -206,12 +206,12 @@ $csv_line[$i] = "\"" . $csv_line[$i] . "\""; } if ($i != count($csv_line) - 1) { - $data = $data . $csv_line[$i] . ","; + $data .= $csv_line[$i] . ","; } else { - $data = $data . $csv_line[$i]; + $data .= $csv_line[$i]; } } - $data = $data . "\n"; + $data .= "\n"; unset($csv_line); } cdr_download($data, $fname);That will speed up execution, it should produce the same output as before.
As for download all, if you to download 100,000 records the memory in apache/php will run out.
I will look into that to see what I can do.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
Okay updated changes. Looks
Okay updated changes. Looks good everything matches up. I still have a few where half the clid goes into src. Further research shows that the clid has a "," in the name field. Thus when the function to separate the csv file is done it thinks it should be a new column.
Example
,
OK, could you extract that
OK, could you extract that line manually from mysql and pm to me so that I can see how it look like.
CDR should enclose all strings with a ',' with '"' during process of the csv.
Instead of open it in Excel/Open Office, save it to a file, then open it in Notepad++ (shameless plugin). Look for the faulty line and see how it look like.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
Chan. Column info
After doing a search in the Return Area.
Is it possible to show more info for Src Chan. or Dst Chan. like SIP/Ext# or Sip/Trunk without hovering over them. Currently mine says SIP unless I hover over it. For incoming calls I use ring groups and the Dest. says the ring group but the Dst Chan. is really the area I'm interested in it shows the ext the pickups the call in the ring group.
Thanks
Ran
There are limited space on
There are limited space on the screen to have that displayed, compare to the old report tool, that displayed it, but it was cut off.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
User interface tweak
In the Call Detail Record Search
The form option buttons are on the left for "Order By" and "Extra options" but the search conditions have them on the right it is confusing, at least to my brain I keep selecting the wrong option. I suggest putting them all on the left.
Thanks
Ran
Fixed. It was confusing.
Fixed. It was confusing.
Mikael Carlsson
(I am off-line, tinkering with my Chevy and my radios, don't know when I will be back)
My dream query request.
I may be missing it but I don't see a way to show the total of call times. Say I want to know how many minutes ext 510 was on for today. Or multiple extensions in one report
The Call Graph by Day appears to do this I could run it for one extension twice. Once for incoming and one for outgoing then add them manually together. Repeat for all the extensions needed.
This is what I'd love to have I'd even be glad to donate some money.
1. Todays Outgoing total call count for a select "channel" or channels with the sum of the "billsec" for each "channel"
2. Todays Incoming total call count for a select "dstchannel" or dstchannels with the sum of the "billsec" for each "dstchannel"
3. Todays Sum of "incoming call count" and "outgoing call count" for Extension with the sum of outgoing_billsec and incoming_billsec
Example csv report output to one file:
-----------
#Outgoing
Extension, Call count, Billable Sec, Billable Minutes
500, 20, 375, 00:06:25
501
502
503
504
505
#Incoming
Extension, Call count, Billable Sec, Billable Minutes
500, 10, 180, 00:03:00
501
502
503
504
505
#Total
Extension, Call count, Billable Sec, Billable Minutes
500, 30, 555, 00:09:25
501
502
503
504
505
-----------
Thanks
Ran
Hi everyone, I just
Hi everyone, I just installed this on my box and seems ok so far so thanks for all the work.
We use gsm for the recording format which i guess is still not supported as the player doesnt seem to work.
Could we just have a link to download the recording instead?
Thanks
Rob
Is there a more robust
Is there a more robust version available like a pro version at $$$
Gary.
No call recording link?
Hi
I have been away for a while and trying to catch up with things, hence the delayed response here. The latest version has taken away the call recording link on my installation? Any ideas where and what to check?
Thanks a lot.
To Add 'Save' option to recordings
Very clunky and maybe not cleanest way, but works for our purposes..
To add a 'Save' option for recordings...
edit cdr_play.php
Replace:
if (isset($file)) { echo("<embed width='100%' type='audio/basic' src='modules/cdr/cdr_audio.php?cdr_file=" .$file. "' width=300, height=25 autoplay=true loop=false></embed><br>"); }With:
if (isset($file)) { echo("<table><tr><td width=100%><embed width='100%' type='audio/basic' src='modules/cdr/cdr_audio.php?cdr_file=" .$file. "' width=300, height=25 autoplay=true loop=false></embed></td><td width=8></td><td><a href='modules/cdr/cdr_audio.php?cdr_file=" .$file. "' style='text-decoration: none;'>Save</a></td></tr></table>"); }