It is currently November 30th, 2023, 7:50 pm

View unanswered posts | View active topics


Board index » Software » SAM Broadcaster

All times are UTC - 6 hours [ DST ]



 Page 1 of 1
  [ 13 posts ] 
  Previous topic | Next topic 
Author Message
Busby
PostPosted: February 14th, 2022, 7:41 am 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
Full error:[CORE] Unhandled exception (Out of memory while expanding memory stream)

I'm sorry to re-visit this problem but I'm hoping someone here may have encountered this before.

I run my station with 2 windows servers, one I call the live server and another backup.

Both servers run 2 copies of SAM Broadcaster, with their own license keys playing different music genres.

Last year I rented a new server running Windows 2019 and installed the latest version of SAM using MYSQL, but every couple of days it would crash with the above error.

After lots of contact with support they were unable to offer a solution, then for some reason it then worked fine, and continues to do so.

I am now replacing my backup server and have again rented a new one with Windows 2019/MYSQL and the problem has returned.

I have again spoken to support and they tell me its an environmental issue.

I have been battling this for weeks now with no end in site and my old server is about to expire.

Does anyone have any suggestions please, or would be prepared to take a look for me please.

I have compared the SAM setups and server configurations and apart from an extra stream, they are identical.

Is there someone who can save me some sleepless nights please.

One server has 32Gb of memory, whilst the other has 64Gb, so its not a memory problem, and Googling the error suggests a Delphi problem and I understand SAM is written in Delphi.

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
DJ Cassio
PostPosted: February 15th, 2022, 10:24 am 
Offline
SVS Member
SVS Member

Joined: May 8th, 2004, 9:00 am
Posts: 10572
Location: Denver, CO
Could be attributed to a memory leak in a PAL script.

_________________
- Rob Oyler, SVS
Image
Top
 Profile  
 
Busby
PostPosted: February 19th, 2022, 7:04 am 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
DJ Cassio (!empty($user->lang['WROTE'])) ? $user->lang['WROTE'] : ucwords(strtolower(str_replace('_', ' ', 'WROTE'))):
Could be attributed to a memory leak in a PAL script.


Many thanks for the answer - sorry for the delay, but I didn't receive an email even though I'm subscribed.

Can you suggest how I could track that down please. According to the logs, the problem happens around every 2 days, but after totally random events, and not immediately after.

For example, an event will run a show, then 30 minutes in I will get the error.

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
DJ Cassio
PostPosted: February 19th, 2022, 9:14 am 
Offline
SVS Member
SVS Member

Joined: May 8th, 2004, 9:00 am
Posts: 10572
Location: Denver, CO
(!empty($user->lang['QUOTE'])) ? $user->lang['QUOTE'] : ucwords(strtolower(str_replace('_', ' ', 'QUOTE'))):
Declaration
property MemoryLeakWarning: Boolean;
Description
Read/Write
Default value is False

If MemoryLeakWarning is set to True it will warn you about possible memory leaks.
PAL scripts do automatic garbage collection when the end of the script is reached. (Even if Loop is set to True)

Garabage collection refers to the process of automatically freeing all objects and memory allocated during the execution of the PAL script.

If you have a script that allocated a lot of objects without freeing those objects, you might run into resource problems an cause SAM Broadcaster to crash.

Turning MemoryLeakWarning on might help you find obejcts that are not properly getting freed during the cource of the script.

EXAMPLE SCRIPT - HOW TO USE MEMORYLEAKWARNING
(!empty($user->lang['QUOTE'])) ? $user->lang['QUOTE'] : ucwords(strtolower(str_replace('_', ' ', 'QUOTE'))):
{­## This script uses a lot of memory, and never frees it.
After running for a few hours this can start causing problems}

PAL.MemoryLeakWarning := True;
var Q : TDataSet;
var D : DateTime;
{­If we made the loop continue for a few hours it will eat up a lot of memory!
For this example we only repeat the loop for 10 seconds}
D := T['+00:00:10'];
repeat
Q := Query('SELECT * FROM songlist ORDER BY date_played DESC LIMIT 10',[],True);
PAL.LockExecution;
Q.First;
while not Q.EOF do
begin
WriteLn(Q['artist']);
Q.Next;
end;

PAL.UnlockExecution;

{­## We *should* call this statement here

Q.Free;
}
until Now > D;


_________________
- Rob Oyler, SVS
Image
Top
 Profile  
 
Busby
PostPosted: February 19th, 2022, 1:59 pm 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
DJ Cassio (!empty($user->lang['WROTE'])) ? $user->lang['WROTE'] : ucwords(strtolower(str_replace('_', ' ', 'WROTE'))):
(!empty($user->lang['QUOTE'])) ? $user->lang['QUOTE'] : ucwords(strtolower(str_replace('_', ' ', 'QUOTE'))):
Declaration
property MemoryLeakWarning: Boolean;
Description
Read/Write
Default value is False

If MemoryLeakWarning is set to True it will warn you about possible memory leaks.
PAL scripts do automatic garbage collection when the end of the script is reached. (Even if Loop is set to True)

Garabage collection refers to the process of automatically freeing all objects and memory allocated during the execution of the PAL script.

If you have a script that allocated a lot of objects without freeing those objects, you might run into resource problems an cause SAM Broadcaster to crash.

Turning MemoryLeakWarning on might help you find obejcts that are not properly getting freed during the cource of the script.

EXAMPLE SCRIPT - HOW TO USE MEMORYLEAKWARNING
(!empty($user->lang['QUOTE'])) ? $user->lang['QUOTE'] : ucwords(strtolower(str_replace('_', ' ', 'QUOTE'))):
{­## This script uses a lot of memory, and never frees it.
After running for a few hours this can start causing problems}

PAL.MemoryLeakWarning := True;
var Q : TDataSet;
var D : DateTime;
{­If we made the loop continue for a few hours it will eat up a lot of memory!
For this example we only repeat the loop for 10 seconds}
D := T['+00:00:10'];
repeat
Q := Query('SELECT * FROM songlist ORDER BY date_played DESC LIMIT 10',[],True);
PAL.LockExecution;
Q.First;
while not Q.EOF do
begin
WriteLn(Q['artist']);
Q.Next;
end;

PAL.UnlockExecution;

{­## We *should* call this statement here

Q.Free;
}
until Now > D;



Many thanks for the script and info.

I ran the script, but no errors..

The SAM is a backup and not live, but when it crashes it continues to run, but with errors.

I have a couple of scripts which run 24/7, like Twittercast, but are event triggered - like track change, so are in a constant wait state, should those be what I should be looking at?


Attachments:
dump.jpg
dump.jpg [ 191.98 KiB | Viewed 9882 times ]

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
DJ Cassio
PostPosted: February 21st, 2022, 9:07 am 
Offline
SVS Member
SVS Member

Joined: May 8th, 2004, 9:00 am
Posts: 10572
Location: Denver, CO
I wasn't suggesting for you to RUN THAT script. It was just an example of where to place the command MemoryLeakWarning in YOUR SCRIPTS!

_________________
- Rob Oyler, SVS
Image
Top
 Profile  
 
Busby
PostPosted: February 21st, 2022, 9:07 am 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
An update from our previous conversation.

I stopped the scripts which are running 24/7 and the problem still occurred again this morning.

I took the opportunity to run the memory leak PAL in the crashed state and there were no errors.

My live server doesn't have this problem and I have checked and double checked that the backup server doesn't have any running processes that the live server doesn't have, and there are none.

One minor difference is that the live server is running an older version of MYSQL and I'm curious if that may be the issue?

Thanks in advance..

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
Busby
PostPosted: February 21st, 2022, 9:23 am 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
DJ Cassio (!empty($user->lang['WROTE'])) ? $user->lang['WROTE'] : ucwords(strtolower(str_replace('_', ' ', 'WROTE'))):
I wasn't suggesting for you to RUN THAT script. It was just an example of where to place the command MemoryLeakWarning in YOUR SCRIPTS!


Sorry, I didn't realise that..

Can you tell me how to amend this script please:
(!empty($user->lang['CODE'])) ? $user->lang['CODE'] : ucwords(strtolower(str_replace('_', ' ', 'CODE'))):
PAL.WaitForTime(T['+00:00:10']);
PAL.LockExecution;
PAL.Loop := True;
var Song : TSongInfo;
var location: string;
location := (ActivePlayer.GetSongInfo['filename']);
//writeln(location);
Delete(location, 17, 100);
//WriteLn('Trimmed file location data');
//writeln(location);
if location= ('C:\Radio\Replay\') then
 begin
  Song := ActivePlayer.GetSongInfo;
  Song['title'] := (Song['title'] + ' [Replay]');
  encoders.SongChange(Song);
  Song.Free;
 end;
PAL.UnlockExecution;
Song.Free;
PAL.WaitForPlayCount(1);

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
Busby
PostPosted: February 22nd, 2022, 8:48 am 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
Another update:
I rented another Windows 2019 server (the fourth so far), and installed a new instance of SAM, but after 5 days of operation, it crashed the same as all previous attempts.

To be honest, I'm at a loss to know where to go next.

I have offered to pay Spacial to trouble-shoot the issue but they aren't interested.

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
DJ Cassio
PostPosted: February 22nd, 2022, 10:06 am 
Offline
SVS Member
SVS Member

Joined: May 8th, 2004, 9:00 am
Posts: 10572
Location: Denver, CO
I would not expect your example of a script to cause an issue since the script free's up it's data variable. However, in order to use MemoryLeakWarning you would insert the command near the top of the script, such as :
(!empty($user->lang['CODE'])) ? $user->lang['CODE'] : ucwords(strtolower(str_replace('_', ' ', 'CODE'))):
PAL.WaitForTime(T['+00:00:10']);
PAL.MemoryLeakWarning := True;
PAL.LockExecution;
PAL.Loop := True;
var Song : TSongInfo;
var location: string;
location := (ActivePlayer.GetSongInfo['filename']);
//writeln(location);
Delete(location, 17, 100);
//WriteLn('Trimmed file location data');
//writeln(location);
if location= ('C:\Radio\Replay\') then
 begin
  Song := ActivePlayer.GetSongInfo;
  Song['title'] := (Song['title'] + ' [Replay]');
  encoders.SongChange(Song);
  Song.Free;
 end;
PAL.UnlockExecution;
Song.Free;
PAL.WaitForPlayCount(1);


In order to SEE any error you MUST run that modified script with the IDE window open as the errors would would be reported in the window at the bottom. You would need to do this individually for every single script you suspect may be having an issue.

_________________
- Rob Oyler, SVS
Image
Top
 Profile  
 
Busby
PostPosted: February 22nd, 2022, 4:55 pm 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
Hi Rob, I've disabled all 3 of the wait-for-event scripts I had running, and also disabled the other scripts run by the ES and still it crashes, so I'm thinking it's not a PAL problem.

I'm presently trying stopping all the encoders (it's a backup player) in the hope of narrowing down what's going on, also comparing MYSQL versions.

If you have any more suggestions I'm all ears..

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
Busby
PostPosted: February 28th, 2022, 9:02 pm 
Offline
Senior User
Senior User

Joined: January 26th, 2006, 7:33 pm
Posts: 196
Location: Douglas, Isle of Man, UK
Hi Rob,

I believe the problem is resolved.

The backup server runs all the time with the encoders trying to connect to the streams in the event the live server goes down.

With the encoders stopped, or connected to available streams, the 2 SAM instances have now run for a week without errors.

My previous SAM 4 instances were happy running like this, but the newer versions obviously aren't.

Thanks for your help and patience.

I will now need to explore 'fallback url' as a solution to my needs.

_________________
Its all about the music..
https://www.1mix.co.uk
Top
 Profile  
 
DJ Cassio
PostPosted: March 2nd, 2022, 9:53 am 
Offline
SVS Member
SVS Member

Joined: May 8th, 2004, 9:00 am
Posts: 10572
Location: Denver, CO
Thanks for the update.

_________________
- Rob Oyler, SVS
Image
Top
 Profile  
 
Display posts from previous:  Sort by  
 Page 1 of 1
  [ 13 posts ] 

Board index » Software » SAM Broadcaster

All times are UTC - 6 hours [ DST ]

Who is online

Users browsing this forum: Bing [Bot] and 23 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group