netbilling down? - GoFuckYourself.com

JerseyPuma · 04-22-2014, 09:29 AM

anyone else having issues? i usually get a call, nothing this time. cant process transactions or access the admin. their www works fine but thats about all. tried calling, just get sent to people's voicemails.

iSpyCams · 04-22-2014, 09:36 AM

I am not able to access the admin, I believe transactions are going through through, my sales and rebills as reported by nats are incrementing as expected including several in the last 30 minutes (actually a little better than expected but not enough to suspect something was wrong)

I am simply not able to access the admin to run reports or process cancellation requests.

I called to talk to support and they said they were on another line and would call me back.

I am 100% sure they are aware of the problem and are fixing it. If I had to guess I would say they are most likely upgrading their SSL setup, since everyone is or should be.

JerseyPuma · 04-22-2014, 09:52 AM

i think it may be intermittent. we are still getting sales but are getting errors back on a lot of others.

iSpyCams · 04-22-2014, 10:05 AM

Quote:

Originally Posted by JerseyPuma

i think it may be intermittent. we are still getting sales but are getting errors back on a lot of others.

How are you aware of the errors if you cant get in the admin?

iSpyCams · 04-22-2014, 10:07 AM

Aaaaand my fucking join page is down

sandman! · 04-22-2014, 10:10 AM

loads fine for me here

iSpyCams · 04-22-2014, 10:12 AM

Quote:

Originally Posted by sandman!

loads fine for me here

what loads? my join page or the netbilling admin?

JerseyPuma · 04-22-2014, 11:31 AM

we don't use hosted join pages. the errors come back when we try to send transactions through the API.

iSpyCams · 04-22-2014, 11:36 AM

Quote:

Originally Posted by JerseyPuma

we don't use hosted join pages. the errors come back when we try to send transactions through the API.

I switched back to my old self hosted join pages and joins are coming in again, I am sure some are failing but no way to tell how many right now till the admin is back up. Maybe not then either.

NETbilling · 04-22-2014, 12:41 PM

Hi All,

NETbilling itself is not down. There is an outage with an interconnecting network backbone, Savvis/Centurylink. It has affected several websites on the internet including the NETbilling Admin Gateway from certain parts of the world. We are in touch with a team of engineers at the data center to get this issue resolved. It extends far beyond just NETbilling.

We will post here when resolved.

Thank you, Mitch Farber

NETbilling · 04-22-2014, 12:42 PM

In the meantime, if you have any questions, please contact [email protected] or call: 661-252-2456.

Thanks, Mitch

NETbilling · 04-22-2014, 12:44 PM

You can see Savvis network status here:

http://internetpulse.com

JerseyPuma · 04-22-2014, 12:50 PM

just got an email back from ben, thanks for the update.

NETbilling · 04-22-2014, 12:53 PM

Quote:

Originally Posted by JerseyPuma

just got an email back from ben, thanks for the update.

Hi,

ok - sent you an email last week and no reply. I may be sending to an old address. Can you email me please with a good address?

Thanks, Mitch

iSpyCams · 04-22-2014, 01:14 PM

Thanks for clarifying Mitch.

mikesouth · 04-22-2014, 09:01 PM

still down for me....

NETbilling · 04-22-2014, 10:08 PM

NETbilling Status

As many are already aware, there have been ongoing network routing issues today between Savvis (our host and network provider) and several of the major internet backbone providers. As a result, many websites and systems, including ours, have been unreachable to a large number of users throughout the day all over the internet. Savvis has indicated that they are performing network-wide maintenance activities to resolve the connectivity and latency issues, which unfortunately, is now directly impacting all NETbilling customers.

We will continue to work with the Savvis team, who are working under a master ticket, and we will bring our systems back up as soon as service is available to us and will notify you as well.

The latest update we have received is that Savvis expects all services to be back to normal by 2am PST.

You can monitor the networks here if you are interested.

If you have any questions, please email: [email protected].

notjoe · 04-23-2014, 01:18 AM

Why do you guys not have multiple uplinks/backbone providers? It is crazy that you would rely on one single provider!

UniqueD · 04-23-2014, 02:52 AM

It seems CenturyLink is downplaying this as scheduled maintenance. Was Netbilling informed of this "scheduled maintenance"?

From Twitter:

Quote:

CenturyLink Tech ‏@CenturyLinkTech 5h
CenturyLink Tech Solutions continues to update Internet routers this evening (US time). May experience brief latency.

Lichen · 04-23-2014, 03:35 AM

Quote:

Originally Posted by pompousjohn

I am simply not able to access the admin to run reports or process cancellation requests.

You are doing it from inside NETbilling admin? Where is it/how's it called?

2MuchMark · 04-23-2014, 04:02 AM

Good luck Mitch!

Quick question: Can you tell me how to read this page? http://internetpulse.com

247mg · 04-23-2014, 04:15 AM

its been over 24 hours now almost all down. no rebill nothing ....

iSpyCams · 04-23-2014, 04:23 AM

Yes, joins halted last night and the admin is still unreachable. This is officially a nightmare.

UniqueD · 04-23-2014, 04:39 AM

I was just able to hit the admin portal for the first time today.

iSpyCams · 04-23-2014, 04:51 AM

same here, yippeeee!

absix · 04-23-2014, 05:38 AM

This situation was handled pretty bad. The email blast should have been sent an hour after you found out what is going on. Not 12 hours later.

Most of us have secondary billers and if we were notified properly this could have minimized the damage for us.

First time I am truly disappointed from Netbilling.

iSpyCams · 04-23-2014, 05:54 AM

Quote:

Originally Posted by absix

This situation was handled pretty bad. The email blast should have been sent an hour after you found out what is going on. Not 12 hours later.

Most of us have secondary billers and if we were notified properly this could have minimized the damage for us.

First time I am truly disappointed from Netbilling.

Let's be fair, Netbilling wasn't down, it just wasn't reachable for most of us. This was basically an ISP outage and I don't think most ISP customer feel they need to email all their customers everytime their ISP has an issue - The same reason Wal-Mart isn't likely to notify you of roadwork or traffic accidents that may affect your ability to shop at their stores. (not a perfect analogy, but similar logic)

Edit: Also, if we couldn't reach them, how are they supposed to reach us? The same issue keeping us out of our admins most likely kept them out unless they were onsite at the datacenter or something.

I doubt Mitch has all our emails in his phone and is able to thumb type notices to us individually. I'm just glad it's over.

247mg · 04-23-2014, 06:13 AM

up and running now

2MuchMark · 04-23-2014, 07:57 AM

Netbilling is back up (and running very fast too!)

John is right - this wasn't Netbilling's fault, and even backup servers would not have made any difference because the problem was a network issue at a datacenter.

Their tech support was perfect. When we noticed the problem and contacted them, their tech support told me what the issue was right away and when we could expect it to be resolved right away.

As for any money lost due to not being able to bill, One thing that you can do next time is have multiple billing providers and then just switch your secondary biller to your primary. This is what we did.

cheers.

NETbilling · 04-23-2014, 10:09 AM

Hi all,

Email notification was sent out within the first hour. And again within 10 minutes if us coming back up at 4 this morning.
We are also updating our Call fire system for phone notifications so please be sure to email [email protected] any updated cell phone numbers that you want calls sent to if there is trouble.

This was the first time in 16 years that this type of issue occurred with Savvis. Their track record and ours are exceptional.

Thank you all for your business.

Rebills are all getting caught up now as well.

Mitch

myjah · 04-23-2014, 03:48 PM

Quote:

Originally Posted by NETbilling

Hi all,

Email notification was sent out within the first hour. And again within 10 minutes if us coming back up at 4 this morning.
We are also updating our Call fire system for phone notifications so please be sure to email [email protected] any updated cell phone numbers that you want calls sent to if there is trouble.

This was the first time in 16 years that this type of issue occurred with Savvis. Their track record and ours are exceptional.

Thank you all for your business.

Rebills are all getting caught up now as well.

Mitch

Good to hear, handled very well.

04-22-2014, 09:29 AM	#1
JerseyPuma Confirmed User Join Date: May 2002 Location: FL Posts: 1,149	netbilling down? anyone else having issues? i usually get a call, nothing this time. cant process transactions or access the admin. their www works fine but thats about all. tried calling, just get sent to people's voicemails. __________________ LOVERCASH.COM good dating sites.

04-22-2014, 09:36 AM	#2
iSpyCams Amateur Gynecologist Industry Role: Join Date: May 2009 Location: Medellin Posts: 4,436	I am not able to access the admin, I believe transactions are going through through, my sales and rebills as reported by nats are incrementing as expected including several in the last 30 minutes (actually a little better than expected but not enough to suspect something was wrong) I am simply not able to access the admin to run reports or process cancellation requests. I called to talk to support and they said they were on another line and would call me back. I am 100% sure they are aware of the problem and are fixing it. If I had to guess I would say they are most likely upgrading their SSL setup, since everyone is or should be. __________________ - As soon as I think up a good sig it's going here.

04-22-2014, 09:52 AM	#3
JerseyPuma Confirmed User Join Date: May 2002 Location: FL Posts: 1,149	i think it may be intermittent. we are still getting sales but are getting errors back on a lot of others. __________________ LOVERCASH.COM good dating sites.

04-22-2014, 10:07 AM	#5
iSpyCams Amateur Gynecologist Industry Role: Join Date: May 2009 Location: Medellin Posts: 4,436	Aaaaand my fucking join page is down __________________ - As soon as I think up a good sig it's going here.

04-22-2014, 10:10 AM	#6
sandman! Icq: 14420613 Industry Role: Join Date: Mar 2001 Location: chicago Posts: 15,432	loads fine for me here __________________ Need WebHosting ? Email me for some great deals [email protected]

04-22-2014, 11:31 AM	#8
JerseyPuma Confirmed User Join Date: May 2002 Location: FL Posts: 1,149	we don't use hosted join pages. the errors come back when we try to send transactions through the API. __________________ LOVERCASH.COM good dating sites.

04-22-2014, 12:41 PM	#10
NETbilling Confirmed User Industry Role: Join Date: Jan 2002 Location: Huntington Beach, CA Posts: 8,580	Hi All, NETbilling itself is not down. There is an outage with an interconnecting network backbone, Savvis/Centurylink. It has affected several websites on the internet including the NETbilling Admin Gateway from certain parts of the world. We are in touch with a team of engineers at the data center to get this issue resolved. It extends far beyond just NETbilling. We will post here when resolved. Thank you, Mitch Farber __________________ Mitch Farber CEO - NETbilling, Inc. Email / Phone: 888-357-8166 / 661-252-2456 Transaction processing & 24/7 call center services with exceptional rates and flexibility, since 1998!

04-22-2014, 12:42 PM	#11
NETbilling Confirmed User Industry Role: Join Date: Jan 2002 Location: Huntington Beach, CA Posts: 8,580	In the meantime, if you have any questions, please contact [email protected] or call: 661-252-2456. Thanks, Mitch __________________ Mitch Farber CEO - NETbilling, Inc. Email / Phone: 888-357-8166 / 661-252-2456 Transaction processing & 24/7 call center services with exceptional rates and flexibility, since 1998!

04-22-2014, 12:44 PM	#12
NETbilling Confirmed User Industry Role: Join Date: Jan 2002 Location: Huntington Beach, CA Posts: 8,580	You can see Savvis network status here: http://internetpulse.com __________________ Mitch Farber CEO - NETbilling, Inc. Email / Phone: 888-357-8166 / 661-252-2456 Transaction processing & 24/7 call center services with exceptional rates and flexibility, since 1998!

04-22-2014, 12:50 PM	#13
JerseyPuma Confirmed User Join Date: May 2002 Location: FL Posts: 1,149	just got an email back from ben, thanks for the update. __________________ LOVERCASH.COM good dating sites.

04-22-2014, 01:14 PM	#15
iSpyCams Amateur Gynecologist Industry Role: Join Date: May 2009 Location: Medellin Posts: 4,436	Thanks for clarifying Mitch. __________________ - As soon as I think up a good sig it's going here.

04-22-2014, 09:01 PM	#16
mikesouth Confirmed User Industry Role: Join Date: Jun 2003 Location: My High Horse Posts: 6,334	still down for me.... __________________ Mike South It's No wonder I took up drugs and alcohol, it's the only way I could dumb myself down enough to cope with the morons in this biz.

04-22-2014, 10:08 PM	#17
NETbilling Confirmed User Industry Role: Join Date: Jan 2002 Location: Huntington Beach, CA Posts: 8,580	NETbilling Status As many are already aware, there have been ongoing network routing issues today between Savvis (our host and network provider) and several of the major internet backbone providers. As a result, many websites and systems, including ours, have been unreachable to a large number of users throughout the day all over the internet. Savvis has indicated that they are performing network-wide maintenance activities to resolve the connectivity and latency issues, which unfortunately, is now directly impacting all NETbilling customers. We will continue to work with the Savvis team, who are working under a master ticket, and we will bring our systems back up as soon as service is available to us and will notify you as well. The latest update we have received is that Savvis expects all services to be back to normal by 2am PST. You can monitor the networks here if you are interested. If you have any questions, please email: [email protected]. __________________ Mitch Farber CEO - NETbilling, Inc. Email / Phone: 888-357-8166 / 661-252-2456 Transaction processing & 24/7 call center services with exceptional rates and flexibility, since 1998!

04-23-2014, 01:18 AM	#18
notjoe Confirmed User Industry Role: Join Date: May 2002 Location: Toronto, Canada Posts: 5,599	Why do you guys not have multiple uplinks/backbone providers? It is crazy that you would rely on one single provider!

04-23-2014, 04:02 AM	#21
2MuchMark Videochat Solutions Industry Role: Join Date: Aug 2004 Location: Canada Posts: 48,717	Good luck Mitch! Quick question: Can you tell me how to read this page? http://internetpulse.com __________________ Custom Software \| Server Management \| Integration and Technology Solutions https://www.2much.net

04-23-2014, 04:15 AM	#22
247mg Yellowplum / 247mg Industry Role: Join Date: Feb 2008 Location: Nicosia, CY Posts: 2,161	its been over 24 hours now almost all down. no rebill nothing .... __________________ 247mg.com - Indian Affiliate Program - Over 50+ Sites To Promote - Monetize Your INDIAN Traffic Today!

04-23-2014, 04:23 AM	#23
iSpyCams Amateur Gynecologist Industry Role: Join Date: May 2009 Location: Medellin Posts: 4,436	Yes, joins halted last night and the admin is still unreachable. This is officially a nightmare. __________________ - As soon as I think up a good sig it's going here.

04-23-2014, 04:39 AM	#24
UniqueD Confirmed User Industry Role: Join Date: Aug 2004 Location: Barcelona Posts: 1,022	I was just able to hit the admin portal for the first time today.

04-23-2014, 04:51 AM	#25
iSpyCams Amateur Gynecologist Industry Role: Join Date: May 2009 Location: Medellin Posts: 4,436	same here, yippeeee! __________________ - As soon as I think up a good sig it's going here.

04-23-2014, 05:38 AM	#26
absix Confirmed User Join Date: Aug 2003 Posts: 484	This situation was handled pretty bad. The email blast should have been sent an hour after you found out what is going on. Not 12 hours later. Most of us have secondary billers and if we were notified properly this could have minimized the damage for us. First time I am truly disappointed from Netbilling. __________________ ICQ: 59eleven5623, ab at zoig dot com

04-23-2014, 06:13 AM	#28
247mg Yellowplum / 247mg Industry Role: Join Date: Feb 2008 Location: Nicosia, CY Posts: 2,161	up and running now __________________ 247mg.com - Indian Affiliate Program - Over 50+ Sites To Promote - Monetize Your INDIAN Traffic Today!

04-23-2014, 07:57 AM	#29
2MuchMark Videochat Solutions Industry Role: Join Date: Aug 2004 Location: Canada Posts: 48,717	Netbilling is back up (and running very fast too!) John is right - this wasn't Netbilling's fault, and even backup servers would not have made any difference because the problem was a network issue at a datacenter. Their tech support was perfect. When we noticed the problem and contacted them, their tech support told me what the issue was right away and when we could expect it to be resolved right away. As for any money lost due to not being able to bill, One thing that you can do next time is have multiple billing providers and then just switch your secondary biller to your primary. This is what we did. cheers. __________________ Custom Software \| Server Management \| Integration and Technology Solutions https://www.2much.net

04-23-2014, 10:09 AM	#30
NETbilling Confirmed User Industry Role: Join Date: Jan 2002 Location: Huntington Beach, CA Posts: 8,580	Hi all, Email notification was sent out within the first hour. And again within 10 minutes if us coming back up at 4 this morning. We are also updating our Call fire system for phone notifications so please be sure to email [email protected] any updated cell phone numbers that you want calls sent to if there is trouble. This was the first time in 16 years that this type of issue occurred with Savvis. Their track record and ours are exceptional. Thank you all for your business. Rebills are all getting caught up now as well. Mitch __________________ Mitch Farber CEO - NETbilling, Inc. Email / Phone: 888-357-8166 / 661-252-2456 Transaction processing & 24/7 call center services with exceptional rates and flexibility, since 1998!