Quantcast
Channel: High Availability (Clustering) forum
Viewing all 5648 articles
Browse latest View live
↧

Create failover cluster in RODC

December 13, 2019, 12:14 am
↧
Search

NIC Teaming or MPIO for SAN in Windows Server 2019

December 6, 2019, 5:58 am
$
0
0
I've been looking online on best practice. I have brand new Windows Server 2019 Datacenter installed on my host machine. I am trying to setup something new (in our environment) which is not new to the world I would like to provide high availability for our storage, to store VMs on a CSV. I have two separate 10 Gbps cards. My question is when I set this up and when setting up iSCSI, should I setup NIC Team in server manager for my storage network or should I have two separate cards with separate IPs and turn on MPIO? I am seeing articles about this but most articles are outdated and talk about Windows Server 2012 R2 when technology was not as mature as it is today.
↧
↧

RDS 2019 HA Setup - Remote Desktop Services 2019 High Availability With FS and SQL clustering.

December 16, 2019, 8:52 am
$
0
0
Hi All, 

I am quite new to RDS, I've installed a 2016 instance on VMware vSphere 6.7 without further issues.

I am trying to install an RDS 2019 instance with HA and cannot find much information on how to setup a shared File System and the SQL Cluster instance. 

Configuring this on VMware will be a different issue but would like to ask if you know of any documentation that would guide me with the File system and SQL server setup.

All I could find was Azure based and none On-Premise based information for this item. 

Any help will be appreciated, 

Thank you in advance, 
Eu
 
↧

Failover VM IP Addressing

December 16, 2019, 5:47 pm
$
0
0
Hi guys, I have a HQ and DR setup with replication enabled. How should i setup the replica IPs. Should there be totally different IPs for the HQ and DR, or the replica can always have the same IP as the HQ and keep changing depending on the site?
↧

Security-Kerberos 4 error when restarting node and when logging in

December 17, 2019, 12:09 am
$
0
0

Hello, because of December updates, there was a security-kerberos 4 error KRB_AP_ERR_MODIFIED.

Two-node cluster with S2D. There are two entries in DNS with node addresses for the computer account Failover cluster virtual network name account.

↧
↧

server which hosting Always-on SQL high availability upgrade to windows server 2012 R2

December 16, 2019, 7:28 pm
$
0
0

Hello 

i have windows servers 2008 Enterprise primary and secondary with Always-on SQL 2012 SP4  high availability so could you please how to achieve to upgrade both servers into windows server 2012 R2.

Many thanks in advance

Ansar Ali

↧

Microsoft Windows Server 2016 (64-bit) -- Error ( ChHoose an Option ) Screen

December 17, 2019, 1:57 am
$
0
0

Hi All,

One of my system getting error like" Choose an Option " error getting, I tried repair os also still same error getting.

 Can any one help on this.

I tried below steps :

sfc /sannow
bootrec /fixboot 
bootrec /fixmbr
Obootrec /Sscan  etc...

Thanks,

Venkata Ramana.

Ramana rao

↧

Cluster Storage Volume - odd NTFS permissions

December 18, 2019, 5:18 pm
$
0
0

Recently, we are seeing new folders created on our cluster storage volumes showing the "incorrect" permissions.

The cluster storage root (C:\ClusterStorage), which is the same on a working and non-working system:

C:\ClusterStorage NT SERVICE\ClusSvc:(F)
                  BUILTIN\Administrators:(W,D,Rc)
                  Everyone:(RX)

An example of the volume folder, which is the same on a working and non-working system:

C:\ClusterStorage\Volume1 NT SERVICE\ClusSvc:(F)
                          BUILTIN\Administrators:(W,D,Rc)
                          Everyone:(RX)

An example of a VM folder (on a non-working system):

C:\ClusterStorage\Volume1\VM1 NT SERVICE\ClusSvc:(F)
                              BUILTIN\Administrators:(W,D,Rc)
                              Everyone:(RX)

An example of a VM folder (on a working system):

C:\ClusterStorage\Volume1\VM2 CREATOR OWNER:(OI)(CI)(IO)(F)
                              NT VIRTUAL MACHINE\Virtual Machines:(R,WD,AD)
                              NT VIRTUAL MACHINE\Virtual Machines:(CI)(IO)(GR,WD,AD)
                              NT AUTHORITY\SYSTEM:(F)
                              BUILTIN\Administrators:(I)(OI)(CI)(F)
                              NT AUTHORITY\SYSTEM:(I)(OI)(CI)(F)
                              CREATOR OWNER:(I)(OI)(CI)(IO)(F)
                              BUILTIN\Users:(I)(OI)(CI)(RX)
                              BUILTIN\Users:(I)(CI)(AD)
                              BUILTIN\Users:(I)(CI)(WD)

I notice that there's a whole bunch of inherited permissions on the working system, which are not present on the non-working system, however these permissionsare not set on the observed parent.

I know that the "folders" in the cluster storage volume folder (C:\ClusterStorage\) are junctions/mount points to the actual NTFS volumes, so I guess that I need to modify the root NTFS permissions of this mounted volume somehow.  Does anyone have an idea on how to do this?


↧
Search

Validate SCSI-3 Persistent Reservation - Presistent Reservation command took longer than 3 seconds

December 19, 2019, 5:11 am
$
0
0

I'm getting the following warning when I validate my cluster nodes.

"The test has detected that the Presistent Reservation command took longer than 3 seconds to complete. This may impact cluster stability."

All other disks test are OK.

Both cluster nodes are vmware virtual machines with RDM disks attached.

My question is how serious is this warning message? I'm unable to find any information about this error message.

↧
↧

Why automatic failover doen't happened on from Node 2 - node 1 ,2-Node cluster , sql / windows server 2016

December 20, 2019, 7:29 am
$
0
0

Hello Team,

System OS : Windows Server 2016 Standard​
​Physical server ​/on-premises / 2-node cluster /Production
​Error(s) Observed : drive went offline ​

Time of occurrence ? Saturday 12-14- 7:48 CST​

Storage : IBM SVC Storage​

Needs an  RCA why the resources did not failover between the clustered nodesNode 2 and Node 1 on the cluster

Note :Time in question is 12/14/2019 7:38 PM CST or 12/15/2019 1:38 UTC.

Node 2 was the active node with ownership of the cluster resources.  The removal of the drive occurred, SQL Services terminated, and the cluster resources did not automatically failover to Node 1 (which resides in a different Data Center location and would have been unaffected by the scheduled SAN upgrade work)

10:18 PM time period is our manual recovery efforts to fail the resources over to Node 1

Cluster Logs of Node 2  :

:2019/12/15-01:37:58.706 INFO  [GUM] Node 2: executing request locally, gumId:133616, my action: /dm/update, # of updates: 1
000013e0.00001570::2019/12/15-01:38:26.289 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] SQL Server component 'io_subsystem' health state has been changed from 'clean' to 'warning' at 2019-12-14 19:38:26.267
000013e0.00001570::2019/12/15-01:38:46.291 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] SQL Server component 'query_processing' health state has been changed from 'clean' to 'warning' at 2019-12-14 19:38:46.270
000013e0.00001570::2019/12/15-01:38:46.291 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] SQL Server component 'io_subsystem' health state has been changed from 'warning' to 'clean' at 2019-12-14 19:38:46.270
00000e9c.00002a18::2019/12/15-01:38:54.009 INFO  [STM]: Got device removal notification
000023fc.00004498::2019/12/15-01:38:54.009 INFO  [RES] Physical Disk: PNP: \\?\MPIO#Disk&Ven_IBM&Prod_2145&Rev_0000#1&7f6ac24&0&3630303530373638303139313031323141303030303030303030303030443632#{53f56307-b6bf-11d0-94f2-00a0c91efb8b} disk disappeared
000023fc.00004498::2019/12/15-01:38:54.038 INFO  [RES] Physical Disk: PNP: \\?\STORAGE#Volume#{de55b03d-c907-11e9-b804-806e6f6e6963}#0000000000100000#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b} volume disappeared
000023fc.00004498::2019/12/15-01:38:54.038 INFO  [RES] Physical Disk: PnpRemoveVolume: Removing volume \\?\STORAGE#Volume#{de55b03d-c907-11e9-b804-806e6f6e6963}#0000000000100000#{53f5630d-b6bf-11d0-94f2-00a0c91efb8b}
000023fc.00004498::2019/12/15-01:38:54.038 INFO  [RES] Physical Disk <DB Engine (E)>: Incrementing resource refcount to 2.
000023fc.00004498::2019/12/15-01:38:54.038 INFO  [RES] Physical Disk <DB Engine (E)>: Incrementing resource refcount to 3.
000023fc.00004498::2019/12/15-01:38:54.038 INFO  [RES] Physical Disk: PNPDEBUG: CM_Unregister_Notification handle 000002653C855C60
000023fc.00004498::2019/12/15-01:38:54.038 INFO  [RES] Physical Disk <DB Engine (E)>: Decrementing resource refcount to 2.
000023fc.000047dc::2019/12/15-01:38:54.065 INFO  [RES] Physical Disk <DB Engine (E)>: Incrementing resource refcount to 3.
000023fc.000047dc::2019/12/15-01:38:54.065 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\Partition0, status 0xc0000034
000023fc.000047dc::2019/12/15-01:38:54.065 ERR   [RES] Physical Disk: PnpUpdateDiskConfigThread: Failed to open device \Device\Harddisk4\Partition0, status 2
000023fc.000047dc::2019/12/15-01:38:54.065 INFO  [RES] Physical Disk <DB Engine (E)>: Decrementing resource refcount to 2.
000023fc.00004498::2019/12/15-01:38:54.065 INFO  [RES] Physical Disk <DB Engine (E)>: Decrementing resource refcount to 1.
000023fc.00003198::2019/12/15-01:38:54.087 ERR   [RHS] Error 2 from ResourceControl 16777617 for resource DB Engine (E).
000013e0.00001570::2019/12/15-01:38:54.313 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] No more diagnostics results
000013e0.00001570::2019/12/15-01:38:54.313 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Discard the pending result sets
000013e0.00001570::2019/12/15-01:38:54.342 ERR   [RES] SQL Server <SQL Server>: [sqsrvres] ODBC Error: [24000] [Microsoft][SQL Server Native Client 11.0]Invalid cursor state (0)
000013e0.00001570::2019/12/15-01:38:54.342 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Diagnostics is stopped
000013e0.00001618::2019/12/15-01:38:54.342 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Online worker helper is stopped
000013e0.00001570::2019/12/15-01:38:54.342 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Disconnect from SQL Server
000013e0.00001570::2019/12/15-01:38:55.501 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] SQL Server was down
000013e0.0000452c::2019/12/15-01:38:57.048 ERR   [RES] SQL Server <SQL Server>: [sqsrvres] Failure detected, SQL Server is down
000013e0.0000452c::2019/12/15-01:38:57.048 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] IsAlive returns FALSE
000013e0.0000452c::2019/12/15-01:38:57.048 WARN  [RHS] Resource SQL Server IsAlive has indicated failure.
00000e9c.00003fd0::2019/12/15-01:38:57.048 INFO  [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'SQL Server', gen(0) result 1/0.
000013e0.0000452c::2019/12/15-01:38:57.048 INFO  [RHS-WER] Scheduling WER ERROR report in 10.000. ReportId d03519cb-a7e4-4e81-8c3e-3a1021532002;
00000e9c.00003fd0::2019/12/15-01:38:57.048 INFO  [RCM] Res SQL Server: Online -> ProcessingFailure( StateUnknown )
00000e9c.00003fd0::2019/12/15-01:38:57.048 INFO  [RCM] TransitionToState(SQL Server) Online-->ProcessingFailure.
00000e9c.00003fd0::2019/12/15-01:38:57.048 INFO  [RCM] rcm::RcmGroup::UpdateStateIfChanged: (SQL Server (MSSQLSERVER), Online --> Pending)
00000e9c.00003fd0::2019/12/15-01:38:57.048 ERR   [RCM] rcm::RcmResource::HandleFailure: (SQL Server)
00000e9c.00003fd0::2019/12/15-01:38:57.049 INFO  [RCM] resource SQL Server: failure count: 0, restartAction: 0 persistentState: 1.
00000e9c.00003fd0::2019/12/15-01:38:57.049 INFO  [RCM] Will queue immediate restart (500 milliseconds) of SQL Server after terminate is complete.

000023fc.0000377c::2019/12/15-01:38:58.387 INFO  [RES] Physical Disk <DB Engine (E)>: Decrementing resource refcount to 1.
00001f24.00004788::2019/12/15-01:38:58.688 INFO  [RES] Generic Service <GxClusPlugIn (shpt19db12) (Instance001)>: GenSvcTerminate : calling SCM (didStop=1)
00001f24.00004788::2019/12/15-01:38:58.689 INFO  [RES] Generic Service <GxClusPlugIn (shpt19db12) (Instance001)>: GenSvcTerminate: retrying...
00000e9c.00001dcc::2019/12/15-01:38:58.829 INFO  [CM] mscs::RegCheckpoint::RegisterForKeyChangeNotifications: Registering for key change notifications for key SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQLServer and resource SQL Network Name (SHPT19DB12).
00000e9c.00001dcc::2019/12/15-01:38:58.886 INFO  [GUM] Node 2: executing request locally, gumId:133617, my action: /dm/update, # of updates: 1
00001f24.00004788::2019/12/15-01:38:59.001 INFO  [RES] Generic Service <GxClusPlugIn (shpt19db12) (Instance001)>: GenSvcTerminate : calling SCM (didStop=1)
00001f24.00004788::2019/12/15-01:38:59.001 INFO  [RES] Generic Service <GxClusPlugIn (shpt19db12) (Instance001)>: GenSvcTerminate: retrying...
00000e9c.00003f64::2019/12/15-01:38:59.252 INFO  [RCM] HandleMonitorReply: TERMINATERESOURCE for 'SQL Server Agent', gen(0) result 0/0.
00000e9c.00003f64::2019/12/15-01:38:59.252 INFO  [RCM] Res SQL Server Agent: [Terminating to OfflineDueToProvider] -> OfflineDueToProvider( StateUnknown )
00000e9c.00003f64::2019/12/15-01:38:59.252 INFO  [RCM] TransitionToState(SQL Server Agent) [Terminating to OfflineDueToProvider]-->OfflineDueToProvider.
00000e9c.00003f64::2019/12/15-01:38:59.252 INFO  [RCM] Res SQL Server: [WaitingToTerminate to OfflineDueToProvider] -> Terminating( OfflineDueToProvider )
00000e9c.00003f64::2019/12/15-01:38:59.252 INFO  [RCM] TransitionToState(SQL Server) [WaitingToTerminate to OfflineDueToProvider]-->[Terminating to OfflineDueToProvider].
000013e0.0000452c::2019/12/15-01:38:59.252 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Request to terminate SQL Server
000013e0.0000452c::2019/12/15-01:38:59.252 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Stop service MSSQLSERVER immediately
000013e0.00001570::2019/12/15-01:38:59.252 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Extended Event logging is stopped
000013e0.0000452c::2019/12/15-01:38:59.252 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Service was stopped successfully
000013e0.00001570::2019/12/15-01:38:59.254 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Extended Event target state:
000013e0.00001570::2019/12/15-01:38:59.254 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Extended Event session summary: dropped buffers = 0, dropped events = 0
000013e0.00001570::2019/12/15-01:38:59.272 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Online worker is stopped
000013e0.0000452c::2019/12/15-01:38:59.273 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] Terminate handling is completed
000013e0.0000452c::2019/12/15-01:38:59.273 INFO  [RES] SQL Server <SQL Server>: [sqsrvres] SQL Server resource state is changed from 'ClusterResourceOnline' to 'ClusterResourceFailed'
000013e0.0000452c::2019/12/15-01:38:59.273 WARN  [RHS] returning ResourceExitStateTerminate.
00000e9c.00003fd0::2019/12/15-01:38:59.273 INFO  [RCM] HandleMonitorReply: TERMINATERESOURCE for 'SQL Server', gen(1) result 0/0.
00000e9c.00003fd0::2019/12/15-01:38:59.273 INFO  [RCM] Res SQL Server: [Terminating to OfflineDueToProvider] -> OfflineDueToProvider( StateUnknown )
00000e9c.00003fd0::2019/12/15-01:38:59.273 INFO  [RCM] TransitionToState(SQL Server) [Terminating to OfflineDueToProvider]-->OfflineDueToProvider.
00000e9c.00003fd0::2019/12/15-01:38:59.273 INFO  [RCM] DB Engine (E) not yet ready to terminate; dependent GxClusPlugIn (shpt19db12) (Instance001) still terminating.
00001f24.00004788::2019/12/15-01:38:59.314 INFO  [RES] Generic Service <GxClusPlugIn (shpt19db12) (Instance001)>: GenSvcTerminate : calling SCM (didStop=1)
00001f24.00004788::2019/12/15-01:38:59.314 INFO  [RES] Generic Service <GxClusPlugIn (shpt19db12) (Instance001)>: GenSvcTerminate: retrying...019/12/15-01:39:03.690 INFO  [RCM] Res GxClusPlugIn (shpt19db12) (Instance001): [Terminating to OfflineDueToProvider] -> OfflineDueToProvider( StateUnknown )

000013e0.000041d8::2019/12/15-01:39:07.247 INFO  [RHS-WER] Capture C:\Windows\Cluster\Reports\CLUSWER_RHS_ERROR_89020180-cdcb-4b69-b97a-62abf26ec554_9.evtx completed.
000013e0.000041d8::2019/12/15-01:39:07.247 INFO  [RHS-WER] Capturing log using query <QueryList><Query Id="0"><Select Path="Microsoft-Windows-ClusterAwareUpdating/Admin">*[System[TimeCreated[timediff(@SystemTime) &lt;= 86400000]]]</Select></Query></QueryList> to C:\Windows\Cluster\Reports\CLUSWER_RHS_ERROR_89020180-cdcb-4b69-b97a-62abf26ec554_10.evtx.
000013e0.000041d8::2019/12/15-01:39:07.271 INFO  [RHS-WER] Capture C:\Windows\Cluster\Reports\CLUSWER_RHS_ERROR_89020180-cdcb-4b69-b97a-62abf26ec554_10.evtx completed.

2019/12/15-01:52:40.681 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\ClusterPartition1, status 0xc0000034
000023fc.00003198::2019/12/15-01:52:40.681 ERR   [RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk4\ClusterPartition1, status 2
000023fc.00003198::2019/12/15-01:52:40.681 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: GetVolumeInformation failed for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
000023fc.00003198::2019/12/15-01:52:40.681 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: failed to get partition size for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
000023fc.00004498::2019/12/15-01:52:40.685 INFO  [RES] Physical Disk: HardDiskpQueryDiskFromStm: ClusterStmFindDisk returned device='\\?\MPIO#Disk&Ven_IBM&Prod_2145&Rev_0000#1&7f6ac24&0&3630303530373638303139313031323141303030303030303030303030443632#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
000023fc.00004498::2019/12/15-01:52:40.685 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\ClusterPartition1, status 0xc0000034
000023fc.00004498::2019/12/15-01:52:40.685 ERR   [RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk4\ClusterPartition1, status 2
000023fc.00004498::2019/12/15-01:52:40.685 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: GetVolumeInformation failed for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
000023fc.00004498::2019/12/15-01:52:40.686 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: failed to get partition size for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3

2019/12/15-01:52:43.451 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\ClusterPartition1, status 0xc0000034
000023fc.000015b4::2019/12/15-01:52:43.451 ERR   [RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk4\ClusterPartition1, status 2
000023fc.000015b4::2019/12/15-01:52:43.451 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: GetVolumeInformation failed for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
000023fc.000015b4::2019/12/15-01:52:43.452 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: failed to get partition size for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
2019/12/15-02:07:43.953 WARN  [RCM] ResourceTypeChaseTheOwnerLoop::DoCall: ResType MSMQ's DLL is not present on this node.  Attempting to find a good node...
00000e9c.00003f64::2019/12/15-02:07:43.966 WARN  [RCM] ResourceTypeChaseTheOwnerLoop::DoCall: ResType MSMQTriggers's DLL is not present on this node.  Attempting to find a good node...
000023fc.00002804::2019/12/15-02:07:43.985 INFO  [RES] Physical Disk: HardDiskpQueryDiskFromStm: ClusterStmFindDisk returned device='\\?\MPIO#Disk&Ven_IBM&Prod_2145&Rev_0000#1&7f6ac24&0&3630303530373638303139313031323141303030303030303030303030443632#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
000023fc.00002804::2019/12/15-02:07:43.986 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\ClusterPartition1, status 0xc0000034

019/12/15-02:22:40.260 INFO  [RES] Network Name <Cluster Name>: Getting Read only private properties
00002140.00002d30::2019/12/15-02:22:40.438 INFO  [RES] Network Name <SHPT19DB12DTC>: Getting Read only private properties
00002140.00002d30::2019/12/15-02:22:40.483 INFO  [RES] Network Name <SQL Network Name (SHPT19DB12)>: Getting Read only private properties

2019/12/15-02:22:40.614 INFO  [RES] Physical Disk: HardDiskpQueryDiskFromStm: ClusterStmFindDisk returned device='\\?\MPIO#Disk&Ven_IBM&Prod_2145&Rev_0000#1&7f6ac24&0&3630303530373638303139313031323141303030303030303030303030443632#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
000023fc.000021dc::2019/12/15-02:22:40.614 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\ClusterPartition1, status 0xc0000034
000023fc.000021dc::2019/12/15-02:22:40.614 ERR   [RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk4\ClusterPartition1, status 2
000023fc.000021dc::2019/12/15-02:22:40.614 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: GetVolumeInformation failed for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
000023fc.000021dc::2019/12/15-02:22:40.614 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: failed to get partition size for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
000023fc.00003198::2019/12/15-02:22:40.619 INFO  [RES] Physical Disk: HardDiskpQueryDiskFromStm: ClusterStmFindDisk returned device='\\?\MPIO#Disk&Ven_IBM&Prod_2145&Rev_0000#1&7f6ac24&0&3630303530373638303139313031323141303030303030303030303030443632#{53f56307-b6bf-11d0-94f2-00a0c91efb8b}'
000023fc.00003198::2019/12/15-02:22:40.619 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\ClusterPartition1, status 0xc00000342019/12/15-03:22:42.522 ERR   [RES] Physical Disk: Failed to open device \Device\Harddisk4\ClusterPartition1, status 0xc0000034
000023fc.000044cc::2019/12/15-03:22:42.522 ERR   [RES] Physical Disk: HarddiskpIsPartitionHidden: failed to open device \Device\Harddisk4\ClusterPartition1, status 2
000023fc.000044cc::2019/12/15-03:22:42.523 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: GetVolumeInformation failed for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
000023fc.000044cc::2019/12/15-03:22:42.523 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: failed to get partition size for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3

Logs after manual failover happened : 

000023fc.00002804::2019/12/15-04:07:44.556 ERR   [RES] Physical Disk: HardDiskpGetDiskInfo: failed to get partition size for \\?\GLOBALROOT\Device\Harddisk4\ClusterPartition1\, status 3
00000e9c.00001238::2019/12/15-04:18:39.550 INFO  [CS] PreShutdown notification.
00000e9c.00001238::2019/12/15-04:18:39.550 INFO  [CS] Service Stopping...
00000e9c.00001238::2019/12/15-04:18:39.550 INFO  [CORE] Node quorum state is 'Successfully formed or joined a cluster'. Form/join status with other nodes is as follows:
00000e9c.00001238::2019/12/15-04:18:39.550 INFO  [NODE] Node 2: Farthest reported progress joining with node SHPT19DB01 (id 1) is: Join Succeeded at time 2019/11/22-05:13:06.752: status 0
00000e9c.000011f0::2019/12/15-04:18:39.550 INFO  [CORE] Graceful shutdown reported by node SHPT19DB02, reason ServiceStopReason::SystemPreShutdown (payload 4)
00000e9c.00001238::2019/12/15-04:18:39.677 INFO  [DCM] UnregisterSwProvider(exit): HrError(0x00000000)
00000e9c.00001238::2019/12/15-04:18:39.678 INFO  [GUM] Node 2: Processing RequestLock 2:32966
00000e9c.00004160::2019/12/15-04:18:39.679 INFO  [GUM] Node 2: Processing GrantLock to 2 (sent by 1 gumid: 133620)
00000e9c.00001238::2019/12/15-04:18:39.679 INFO  [GUM] Node 2: executing request locally, gumId:133621, my action: qm/set-node-weight, # of updates: 1
00000e9c.00001238::2019/12/15-04:18:39.679 WARN  [QUORUM] Node 2: weight adjustment not performed. Cannot go below weight count 3 in a hybrid configuration with 2+ nodes
00000e9c.00001238::2019/12/15-04:18:39.680 INFO  [DM] An empty single transaction is cancelled 27:27:146656+1::0
00000e9c.00001238::2019/12/15-04:18:39.683 INFO  [GUM] Node 2: executing request locally, gumId:133622, my action: /dm/update, # of updates: 1
00000e9c.00001fec::2019/12/15-04:18:39.687 INFO  [RCM] rcm::RcmAgent::Unload: [RCM] Shutdown lock acquired, proceeding, initPhase:false StopReason:ServiceStopReason::SystemPreShutdown
00000e9c.00001fec::2019/12/15-04:18:39.687 INFO  [RCM] rcm::RcmAgent::Unload: [RCM] Director node is shutting down, choose new director
00000e9c.00001fec::2019/12/15-04:18:39.687 INFO  [RCM] Sending causal to set new director node 1
00000e9c.00001fec::2019/12/15-04:18:39.687 INFO  [RCM] Director is set to node 1
00000e9c.00001fec::2019/12/15-04:18:39.688 INFO  [RCM-ClusterInit] Cluster initialization done using older create cluster mechanism
00000e9c.00001fec::2019/12/15-04:18:39.688 INFO  [RCM-ClusterInit] Not queuing work item to initialize cluster core as it has already been initialized.
00000e9c.00001fec::2019/12/15-04:18:39.688 INFO  [RCM] rcm::DrainMgr::DrainNode: [DrainMgr] Initiating DrainNode
00000e9c.00001f18::2019/12/15-04:18:39.696 INFO  [RCM] Waiting controls thread detected Rcm shutting down, exiting
00000e9c.00001f18::2019/12/15-04:18:39.696 INFO  [RCM] Waiting controls processing thread is shutting down and draining any remaining controls it was monitoring.
00000e9c.00001fec::2019/12/15-04:18:39.726 INFO  [RCM] rcm::DrainMgr::ValidateDrainNode: [DrainMgr] Node state:Up. DrainStatus:0. bServiceShutdown:1
00000e9c.00001fec::2019/12/15-04:18:39.726 INFO  [RCM] rcm::DrainMgr::StartDrain: [DrainMgr] Starting Drain.
00000e9c.00001fec::2019/12/15-04:18:39.726 INFO  [RCM] rcm::DrainMgr::UpdateDrainStatus: [DrainMgr] Changing DrainStatus for node "2" to 1
00000e9c.00001fec::2019/12/15-04:18:39.726 INFO  [GUM] Node 2: executing request locally, gumId:133623, my action: /dm/update, # of updates: 1
00000e9c.00001fec::2019/12/15-04:18:39.743 INFO  [RCM] rcm::DrainMgr::UpdateDrainTarget: [DrainMgr] Changing DrainTarget for node to "-1"
00000e9c.00001fec::2019/12/15-04:18:39.743 INFO  [RCM] rcm::DrainMgr::StartDrain: [DrainMgr] Doing Drain without setting to paused state
00000e9c.00001fec::2019/12/15-04:18:39.752 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (Cluster Group)
00000e9c.00001fec::2019/12/15-04:18:39.752 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (Cluster Group)
00000e9c.00001fec::2019/12/15-04:18:39.752 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] Initiating MoveGroupForDrain for group:Cluster Group with Flags:0, move count:0
00000e9c.00001fec::2019/12/15-04:18:39.752 INFO  [RCM-plcmt] This node is not director, node 1 is.  Asking others for placement...
00000e9c.00001fec::2019/12/15-04:18:39.752 INFO  [RCM-plcmt] asking node 1 placement decision, attempt 1
00000e9c.00001f18::2019/12/15-04:18:39.755 ERR   [API] s_ApiGetNotifyAsync: (5073)' because of 'Shutting down.'
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  [RCM-plcmt] done waiting...
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  [RCM-plcmt] Node 1 replied to placement request g=Cluster Group tgt=1 wait=false
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  MTimer(GetPlacementFromDirector): [Start to Multitimer_destroyed : 16 ms]
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  MTimer(GetPlacementFromDirector): [Total: 16 ms ( 0 s )]
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  [RCM] placement manager picked node 1 for drain move of group Cluster Group
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] calling queueGroup:Cluster Group with flags:4 operationType:1 bBounceBack:0
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  [RCM] rcm::RcmGroup::QueueGroup: (Cluster Group, 1, 4)
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  [RCM] Cluster Group: Added Flags 4 to StatusInformation. New StatusInformation 4
00000e9c.00001fec::2019/12/15-04:18:39.763 INFO  [RCM] rcm::QueuedMovesHolder::AddGroup: Adding group Cluster Group)
00000e9c.000034f0::2019/12/15-04:18:39.763 ERR   [API] s_ApiGetNotifyAsync: (5073)' because of 'Shutting down.'
00000e9c.00001f18::2019/12/15-04:18:39.764 ERR   [API] s_ApiGetNotifyAsync: (5073)' because of 'Shutting down.'
00002140.00002f9c::2019/12/15-04:18:39.765 ERR   [RES] Network Name <Cluster Name>: Error: 0x800706BA in FileServerDefaultShareImpl::ResyncDisks@352.
00002140.00002f9c::2019/12/15-04:18:39.765 ERR   [RES] Network Name <Cluster Name>: Error: 0x800706BA in FileServerDefaultShareImpl::NotificationThread@528.
00000e9c.00001fec::2019/12/15-04:18:39.767 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] returning from MoveGroupForDrain for group:Cluster Group with status:0
00000e9c.00001fec::2019/12/15-04:18:39.767 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (SHPT19DB12DTC)
00000e9c.00001fec::2019/12/15-04:18:39.767 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (SHPT19DB12DTC)
00000e9c.00001fec::2019/12/15-04:18:39.767 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] Initiating MoveGroupForDrain for group:SHPT19DB12DTC with Flags:0, move count:0
00000e9c.00001f18::2019/12/15-04:18:39.767 INFO  [RCM-plcmt] This node is not director, node 1 is.  Asking others for placement...
00000e9c.00001fec::2019/12/15-04:18:39.767 INFO  [RCM-plcmt] This node is not director, node 1 is.  Asking others for placement...
00000e9c.00001f18::2019/12/15-04:18:39.767 INFO  [RCM-plcmt] asking node 1 placement decision, attempt 1
00000e9c.00001fec::2019/12/15-04:18:39.767 INFO  [RCM-plcmt] asking node 1 placement decision, attempt 1
00000e9c.00001f18::2019/12/15-04:18:39.770 INFO  [RCM-plcmt] done waiting...
00000e9c.00001f18::2019/12/15-04:18:39.770 INFO  [RCM-plcmt] Node 1 replied to placement request g=Cluster Group tgt=1 wait=false
00000e9c.00001f18::2019/12/15-04:18:39.770 INFO  MTimer(GetPlacementFromDirector): [Total: 0 ms ( 0 s )]
00000e9c.00001f18::2019/12/15-04:18:39.770 INFO  [RCM] placement manager picked node 1 for MoveType::Drain move of group Cluster Group
00000e9c.000034f0::2019/12/15-04:18:39.770 INFO  [RCM] rcm::RcmGroup::DoQueuedMove: (Cluster Group)
00000e9c.000034f0::2019/12/15-04:18:39.770 INFO  [RCM] rcm::RcmGroup::Move: (Cluster Group, 1, MoveType::Drain)
00000e9c.000034f0::2019/12/15-04:18:39.770 INFO  [RCM] rcm::RcmGroup::Move: Bringing group 'Cluster Group' offline first...
00000e9c.000034f0::2019/12/15-04:18:39.770 INFO  [RCM] Res Cluster Name: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.770 INFO  [RCM] TransitionToState(Cluster Name) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] rcm::RcmGroup::UpdateStateIfChanged: (Cluster Group, Online --> Pending)
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] Res Cluster Name: WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] TransitionToState(Cluster Name) WaitingToGoOffline-->OfflineCallIssued.
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] Res Cluster IP Address: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] TransitionToState(Cluster IP Address) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] 'Cluster IP Address' cannot go offline yet; Dependent Resource 'Cluster Name' is in state OfflineCallIssued.
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] Res Cluster IP Address: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.771 INFO  [RCM] rcm::RcmGroup::Offline: deferring offline of quorum resource 'File Share Witness' until all other resources are offline.
00000e9c.00004128::2019/12/15-04:18:39.771 ERR   [API] s_ApiGetNotifyAsync: (5073)' because of 'Shutting down.'
00002140.000017f0::2019/12/15-04:18:39.771 INFO  [RES] Network Name <Cluster Name>: Offline called for resource Cluster Name
00002140.000045c4::2019/12/15-04:18:39.771 INFO  [RES] Network Name <Cluster Name>: Entering Offline thread
00000e9c.00003f20::2019/12/15-04:18:39.771 INFO  [RCM] HandleMonitorReply: OFFLINERESOURCE for 'Cluster Name', gen(0) result 997/0.
00000e9c.00003f20::2019/12/15-04:18:39.771 INFO  [RCM] Res Cluster Name: OfflineCallIssued -> OfflinePending( StateUnknown )
00000e9c.00003f20::2019/12/15-04:18:39.771 INFO  [RCM] TransitionToState(Cluster Name) OfflineCallIssued-->OfflinePending.
00002140.000045c4::2019/12/15-04:18:39.771 INFO  [RES] Network Name <Cluster Name>: Performing actual offline of resource.
00002140.000045c4::2019/12/15-04:18:39.771 INFO  [RES] Network Name: Agent: Closing (1b9cd52e-c9ce-445b-a6cb-30a3b0f86d69,AdminShare)
00002140.000017f0::2019/12/15-04:18:39.771 INFO  [RES] Network Name: Agent: OnClose (1b9cd52e-c9ce-445b-a6cb-30a3b0f86d69,AdminShare)
00002140.000017f0::2019/12/15-04:18:39.771 INFO  [RES] Network Name <Cluster Name>: AdminShare: Canceling work, state: Closing/Idle
00002140.000017f0::2019/12/15-04:18:39.771 INFO  [RES] Network Name <Cluster Name>: AdminShare: OnCloseBase, previous state: Initialized/Idle
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM-plcmt] done waiting...
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM-plcmt] Node 1 replied to placement request g=SHPT19DB12DTC tgt=1 wait=false
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  MTimer(GetPlacementFromDirector): [Total: 0 ms ( 0 s )]
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] placement manager picked node 1 for drain move of group SHPT19DB12DTC
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] calling queueGroup:SHPT19DB12DTC with flags:4 operationType:1 bBounceBack:0
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] rcm::RcmGroup::QueueGroup: (SHPT19DB12DTC, 1, 4)
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] SHPT19DB12DTC: Added Flags 4 to StatusInformation. New StatusInformation 4
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] rcm::QueuedMovesHolder::AddGroup: Adding group SHPT19DB12DTC)
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] returning from MoveGroupForDrain for group:SHPT19DB12DTC with status:0
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (SQL Server (MSSQLSERVER))
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (SQL Server (MSSQLSERVER))
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] Initiating MoveGroupForDrain for group:SQL Server (MSSQLSERVER) with Flags:0, move count:0
00000e9c.00001f18::2019/12/15-04:18:39.773 INFO  [RCM-plcmt] This node is not director, node 1 is.  Asking others for placement...
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM-plcmt] This node is not director, node 1 is.  Asking others for placement...
00000e9c.00001f18::2019/12/15-04:18:39.773 INFO  [RCM-plcmt] asking node 1 placement decision, attempt 1
00000e9c.00001fec::2019/12/15-04:18:39.773 INFO  [RCM-plcmt] asking node 1 placement decision, attempt 1
00000e9c.00003f20::2019/12/15-04:18:39.773 ERR   [API] s_ApiGetNotifyAsync: (5073)' because of 'Shutting down.'
00000e9c.00001f18::2019/12/15-04:18:39.776 INFO  [RCM-plcmt] done waiting...
00000e9c.00001f18::2019/12/15-04:18:39.776 INFO  [RCM-plcmt] Node 1 replied to placement request g=SHPT19DB12DTC tgt=1 wait=false
00000e9c.00001f18::2019/12/15-04:18:39.776 INFO  MTimer(GetPlacementFromDirector): [Start to Multitimer_destroyed : 16 ms]
00000e9c.00001f18::2019/12/15-04:18:39.776 INFO  MTimer(GetPlacementFromDirector): [Total: 16 ms ( 0 s )]
00000e9c.00001f18::2019/12/15-04:18:39.776 INFO  [RCM] placement manager picked node 1 for MoveType::Drain move of group SHPT19DB12DTC
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] rcm::RcmGroup::DoQueuedMove: (SHPT19DB12DTC)
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] rcm::RcmGroup::Move: (SHPT19DB12DTC, 1, MoveType::Drain)
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] rcm::RcmGroup::Move: Bringing group 'SHPT19DB12DTC' offline first...
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res SHPT19DB12DTC: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] TransitionToState(SHPT19DB12DTC) Online-->WaitingToGoOffline.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] rcm::RcmGroup::UpdateStateIfChanged: (SHPT19DB12DTC, Online --> Pending)
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Bringing dependent resource 'New Distributed Transaction Coordinator' offline before provider resource 'SHPT19DB12DTC'.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res New Distributed Transaction Coordinator: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] TransitionToState(New Distributed Transaction Coordinator) Online-->WaitingToGoOffline.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res New Distributed Transaction Coordinator: WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] TransitionToState(New Distributed Transaction Coordinator) WaitingToGoOffline-->OfflineCallIssued.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] 'SHPT19DB12DTC' cannot go offline yet; Dependent Resource 'New Distributed Transaction Coordinator' is in state OfflineCallIssued.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res SHPT19DB12DTC: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res MS DTC: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] TransitionToState(MS DTC) Online-->WaitingToGoOffline.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] 'MS DTC' cannot go offline yet; Dependent Resource 'New Distributed Transaction Coordinator' is in state OfflineCallIssued.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res MS DTC: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res IP Address 10.126.5.129: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] TransitionToState(IP Address 10.126.5.129) Online-->WaitingToGoOffline.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Bringing dependent resource 'SHPT19DB12DTC' offline before provider resource 'IP Address 10.126.5.129'.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] 'SHPT19DB12DTC' cannot go offline yet; Dependent Resource 'New Distributed Transaction Coordinator' is in state OfflineCallIssued.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res SHPT19DB12DTC: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] 'IP Address 10.126.5.129' cannot go offline yet; Dependent Resource 'SHPT19DB12DTC' is in state WaitingToGoOffline.
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res IP Address 10.126.5.129: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] Res cmd Application: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.776 INFO  [RCM] TransitionToState(cmd Application) Online-->WaitingToGoOffline.
00000e9c.00004128::2019/12/15-04:18:39.777 INFO  [RCM] Bringing dependent resource 'MS DTC' offline before provider resource 'cmd Application'.
00000e9c.00004128::2019/12/15-04:18:39.777 INFO  [RCM] 'MS DTC' cannot go offline yet; Dependent Resource 'New Distributed Transaction Coordinator' is in state OfflineCallIssued.
00000e9c.00004128::2019/12/15-04:18:39.777 INFO  [RCM] Res MS DTC: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
00000e9c.00004128::2019/12/15-04:18:39.777 INFO  [RCM] 'cmd Application' cannot go offline yet; Dependent Resource 'MS DTC' is in state WaitingToGoOffline.
00000e9c.00004128::2019/12/15-04:18:39.777 INFO  [RCM] Res cmd Application: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
00001f2c.00003db8::2019/12/15-04:18:39.777 INFO  [RES] Distributed Transaction Coordinator <New Distributed Transaction Coordinator>: 12-14-2019 22:18:39:774 : [1f2c.3db8] 0x00000000 [TRACE_RESOURCE] [  TRACE_INFO] DtcOffline (com\complus\dtc\shared\mtxclu\src\dtcresource.cpp@790): Taking DTC resource offline
00001f2c.00003db8::2019/12/15-04:18:39.777 INFO  [RES] Distributed Transaction Coordinator <New Distributed Transaction Coordinator>: 12-14-2019 22:18:39:774 : [1f2c.3db8] 0x00000000 [TRACE_RESOURCE] [  TRACE_INFO] DtcOffline (com\complus\dtc\shared\mtxclu\src\dtcresource.cpp@798): DTC offline thread successfully started
00000e9c.000034f0::2019/12/15-04:18:39.777 INFO  [RCM] HandleMonitorReply: OFFLINERESOURCE for 'New Distributed Transaction Coordinator', gen(0) result 997/0.
00000e9c.000034f0::2019/12/15-04:18:39.777 INFO  [RCM] Res New Distributed Transaction Coordinator: OfflineCallIssued -> OfflinePending( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.777 INFO  [RCM] TransitionToState(New Distributed Transaction Coordinator) OfflineCallIssued-->OfflinePending.
00001f2c.000021d8::2019/12/15-04:18:39.777 INFO  [RES] Distributed Transaction Coordinator <New Distributed Transaction Coordinator>: 12-14-2019 22:18:39:774 : [1f2c.21d8] 0x00000000 [TRACE_RESOURCE] [  TRACE_INFO] DtcOfflineThread (com\complus\dtc\shared\mtxclu\src\dtcresource.cpp@822): DTC offline thread is running
00001f2c.000021d8::2019/12/15-04:18:39.777 INFO  [RES] Distributed Transaction Coordinator <New Distributed Transaction Coordinator>: 12-14-2019 22:18:39:774 : [1f2c.21d8] 0x00000000 [TRACE_RESOURCE] [  TRACE_INFO] DtcOfflineThread (com\complus\dtc\shared\mtxclu\src\dtcresource.cpp@837): Stopping service...
00002140.00000b98::2019/12/15-04:18:39.778 ERR   [RES] Network Name <SHPT19DB12DTC>: Error: 0x800706BA in FileServerDefaultShareImpl::ResyncDisks@352.
00002140.00000b98::2019/12/15-04:18:39.778 ERR   [RES] Network Name <SHPT19DB12DTC>: Error: 0x800706BA in FileServerDefaultShareImpl::NotificationThread@528.
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM-plcmt] done waiting...
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM-plcmt] Node 1 replied to placement request g=SQL Server (MSSQLSERVER) tgt=1 wait=false
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  MTimer(GetPlacementFromDirector): [Start to Multitimer_destroyed : 16 ms]
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  MTimer(GetPlacementFromDirector): [Total: 16 ms ( 0 s )]
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] placement manager picked node 1 for drain move of group SQL Server (MSSQLSERVER)
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] calling queueGroup:SQL Server (MSSQLSERVER) with flags:4 operationType:1 bBounceBack:0
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] rcm::RcmGroup::QueueGroup: (SQL Server (MSSQLSERVER), 1, 4)
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] SQL Server (MSSQLSERVER): Added Flags 4 to StatusInformation. New StatusInformation 4
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] rcm::QueuedMovesHolder::AddGroup: Adding group SQL Server (MSSQLSERVER))
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] returning from MoveGroupForDrain for group:SQL Server (MSSQLSERVER) with status:0
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (Available Storage)
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] rcm::RcmGroup::GetGroupMoveContext: (Available Storage)
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] Initiating MoveGroupForDrain for group:Available Storage with Flags:0, move count:0
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM-plcmt] This node is not director, node 1 is.  Asking others for placement...
00000e9c.00001fec::2019/12/15-04:18:39.781 INFO  [RCM-plcmt] asking node 1 placement decision, attempt 1
00000e9c.00001f18::2019/12/15-04:18:39.781 INFO  [RCM-plcmt] This node is not director, node 1 is.  Asking others for placement...
00000e9c.00001f18::2019/12/15-04:18:39.781 INFO  [RCM-plcmt] asking node 1 placement decision, attempt 1
00000e9c.000034f0::2019/12/15-04:18:39.781 ERR   [API] s_ApiGetNotifyAsync: (5073)' because of 'Shutting down.'
00000e9c.00002d8c::2019/12/15-04:18:39.782 WARN  [RCM] rcm::ChaseTheOwnerLoop::NoLockIsCallComplete: RCM is shutting down. Not chasing the owner on error 0.
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM-plcmt] done waiting...
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM-plcmt] Node 1 replied to placement request g=Available Storage tgt=1 wait=false
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  MTimer(GetPlacementFromDirector): [Total: 0 ms ( 0 s )]
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] placement manager picked node 1 for drain move of group Available Storage
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] calling groupMove:Available Storage with flags:0 operationType:1 bBounceBack:0
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] rcm::RcmGroup::Move: (Available Storage, 1, MoveType::Drain)
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] No need to bring group 'Available Storage' offline before move; state Offline, owner 2.
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::MoveGroupForDrain: [DrainMgr] returning from MoveGroupForDrain for group:Available Storage with status:0
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::StartDrain: [DrainMgr] StartDrain returned with Status:0
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::DrainNode: [DrainMgr] DrainNode returned with Status:0
00000e9c.00001fec::2019/12/15-04:18:39.783 INFO  [RCM] rcm::RcmAgent::DrainOnShutdown: [Shutdown] Returned from DrainNode. TimeElapsed:94
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  [RCM] rcm::QueuedMovesHolder::RemoveGroup: (Available Storage) GroupBeingMoved: true AllowMoveCancel: false NotifyMoveFailure: false
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  [GUM] Node 2: executing request locally, gumId:133624, my action: /rcm/gum/GroupMoveOperation, # of updates: 1
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  [RCM] rcm::RcmGum::GroupMoveOperation(1)
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  [RCM] move of group Available Storage from SHPT19DB02(2) to SHPT19DB01(1) of type MoveType::Drain is about to succeed, failoverCount=0, lastFailoverTime=1601/01/01-00:00:00.000 targeted=false
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::QueueMoveEvent: [DrainMgr] Queued GroupMoveEvent for Group Available Storage
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  MTimer(OneGroupMovetimer): [Total: 0 ms ( 0 s )]
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  [RCM] processed 1 group moves, 0 failures
00000e9c.00002d8c::2019/12/15-04:18:39.783 INFO  [DM] An empty single transaction is cancelled 27:27:146658+1::0
00000e9c.00003f20::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::ProcessGroupMoveEvent: [DrainMgr] Processing GroupMoveEvent for Group Available Storage
00000e9c.00003f20::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::GroupOwnershipChangedWorker: [DrainMgr] Handling Group owner changed Group Available Storage src:2 dest:1 MoveType:MoveType::Drain.
00000e9c.00003f20::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::GroupOwnershipChangedWorker: [DrainMgr] Move of group Available Storage completed from src to 1.
00000e9c.00003f20::2019/12/15-04:18:39.783 INFO  [RCM] rcm::DrainMgr::GroupMoveSucceeded: [DrainMgr] Group move succeeded for Available Storage
00000e9c.00001f18::2019/12/15-04:18:39.788 INFO  [RCM-plcmt] done waiting...
00000e9c.00001f18::2019/12/15-04:18:39.788 INFO  [RCM-plcmt] Node 1 replied to placement request g=SQL Server (MSSQLSERVER) tgt=1 wait=false
00000e9c.00001f18::2019/12/15-04:18:39.788 INFO  MTimer(GetPlacementFromDirector): [Total: 0 ms ( 0 s )]
00000e9c.00001f18::2019/12/15-04:18:39.788 INFO  [RCM] placement manager picked node 1 for MoveType::Drain move of group SQL Server (MSSQLSERVER)
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] rcm::RcmGroup::DoQueuedMove: (SQL Server (MSSQLSERVER))
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] rcm::RcmGroup::Move: (SQL Server (MSSQLSERVER), 1, MoveType::Drain)
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] rcm::RcmGroup::Move: Bringing group 'SQL Server (MSSQLSERVER)' offline first...
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Backups (I): Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Backups (I)) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] rcm::RcmGroup::UpdateStateIfChanged: (SQL Server (MSSQLSERVER), Failed --> Pending)
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Backups (I): WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Backups (I)) WaitingToGoOffline-->OfflineCallIssued.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Temp (H): Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Temp (H)) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Temp (H): WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Temp (H)) WaitingToGoOffline-->OfflineCallIssued.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res MMFO_SHPT19DB12: Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(MMFO_SHPT19DB12) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Bringing dependent resource 'DB Logs (G)' offline before provider resource 'MMFO_SHPT19DB12'.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Logs (G): Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Logs (G)) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Bringing dependent resource 'DB Data (F)' offline before provider resource 'MMFO_SHPT19DB12'.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Data (F): Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Data (F)) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Data (F): WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Data (F)) WaitingToGoOffline-->OfflineCallIssued.
000023fc.00002804::2019/12/15-04:18:39.788 INFO  [RES] Physical Disk <DB Backups (I)>: Offline request.
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] Res DB Logs (G): WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.788 INFO  [RCM] TransitionToState(DB Logs (G)) WaitingToGoOffline-->OfflineCallIssued.
00000e9c.000034f0::2019/12/15-04:18:39.789 INFO  [RCM] 'MMFO_SHPT19DB12' cannot go offline yet; Dependent Resource 'DB Temp (H)' is in state OfflineCallIssued.
00000e9c.000034f0::2019/12/15-04:18:39.789 INFO  [RCM] Res MMFO_SHPT19DB12: WaitingToGoOffline -> WaitingToGoOffline( StateUnknown )
000023fc.00004494::2019/12/15-04:18:39.789 INFO  [RES] Physical Disk <DB Temp (H)>: Offline request.
00000e9c.000034f0::2019/12/15-04:18:39.789 INFO  [RCM] Res SQL Network Name (SHPT19DB12): Online -> WaitingToGoOffline( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.789 INFO  [RCM] TransitionToState(SQL Network Name (SHPT19DB12)) Online-->WaitingToGoOffline.
00000e9c.000034f0::2019/12/15-04:18:39.789 INFO  [RCM] Res SQL Network Name (SHPT19DB12): WaitingToGoOffline -> OfflineCallIssued( StateUnknown )
00000e9c.000034f0::2019/12/15-04:18:39.789 INFO  [RCM] TransitionToState(SQL Network Name (SHPT19DB12)) 

Could you let me know why the automatic failover is not happened from node 2 - node 1 @ 7:38 PM 12/14/2019

Let me know if you need any details required.

Thanking you 

swathi

↧

Setup cannot continue because your cluster is not be installed under a domain

December 22, 2019, 10:20 pm
$
0
0
while application installation windows cluster throws error " Setup cannot continue because your cluster is not be installed under a domain"  systems already in domain. 
↧

No option to add Failover Cluster Manager on Nodes that are already part of a Cluster?

December 23, 2019, 1:21 am
$
0
0

Hello,

I have started a new role where I have inherited a network that I am still trying to learn.

There are 3 nodes that are part of a cluster. Virtual Machine Manager is being used to manage the Cluster. I have installed RSAT on my laptop and can connect to the cluster but on all 3 nodes, when I load MMC, I don't have the option to add the Failover Cluster Manager snap in?

The Failover Cluster feature is installed and the nodes are working fine but I just want to me able to manage them all separately direct from the physical Node itself should I get an issue with Virtual Machine Manager.

It's probably something really obvious i'm missing but I did some research and can't work out what i'm doing wrong!

Help would be much appreciated.

Kind Regards

Gary

↧

Windows Fail-over Cluster High Availability

December 24, 2019, 10:43 am
$
0
0

I have 03 Physical Servers names "Host1" "Host2" and "Host3 . All 03 Servers have same architecture and same configuration and residing on same SITE with same IP subnet.

"Host1" is having Windows server 2016 installed with Hyper-V role installed and hosting VM of Primary Domain Controller named "DC01" and an application server VM named "APP1". Both VM's i.e DC01 and APP1 have Windows Server 2012 installed .

Host2 have Server 2016 Installed with Hyper-V role installed and hosting VM "SQL2012" which is acting as Database server of "APP1

Host3 is newly purchased server and noting installed in it . And Newly purchased SAN is also available .

now i want High available environment of DC01,App1 and SQL2012 for this I am planning to use Windows Failover Clustering . I want to know below Questions.

Question 1: Can I use Windows Fail-over Clustering for VM of "DC01" for High availability.

Question 2 : Can I Use Hyper-V VM replication for "DC01" between "Host1" and "Host3" servers

Question 3 : Can i use Windows Fail-over Clustering for VM of  "SQL2012" High availability.

Regards

AA

↧
↧

0x8007174b failed to bring cluster disk online MD1420 SAS HBA on Windows server 2016 + 2019

December 26, 2019, 4:18 pm
$
0
0

I'm trying to move the storage pool ownership to the 03 node. But it won't accept i:
Error
The operation has failed.
The action 'Move' did not complete.
Error Code: 0x8007174b
Clustered storage is not connected to the node

The setup is 3x Dell R630 with a MD1420 directly attached with HBA SAS cables.

The two working nodes use

Dell 12Gbps HBA

While the node not working uses the PERC H830

All with latest firmware and drivers from lifecycle controller and dell command update.

↧

All VM down in hyper v

December 28, 2019, 7:54 am
$
0
0

Below is my network Scenario,

2 Windows 2012 Hyper vhost (HP server) configured cluster manager. 

HP SAN configured as data server.

4 virtual machines including Domain controller and DNS server. ( DC 1 AND DC 2)

Yesterday Upgraded Avast Antivirus on both the host. After some time none of the users were able to browse the internet or log on to their computer. Up on checking I found that all the VM are in failed state. As the DNS server is one of the failed VM the internet is also down. I have tried restarting HP server and SAN but none of the VM is listed in the hyper manager. In cluster manager I can see both the nodes online. I am able to ping from host 1 to host 2. Not sure what to do now?  Please help

↧
Search

Failover Cluster Manager bug on Server 2019 after .NET 4.8 installed - unable to type more than two characters in to the IP fields

September 20, 2019, 8:46 am
$
0
0

We ran into a nasty bug on Windows Server 2019 and I can't find any KB articles on it. It's really easy to replicate. 

1. Install Windows Server 2019 Standard with Desktop Experience from an ISO. 

2. Install Failover Cluster Services.

3. Create new cluster, on the 4th screen, add the current server name. This is what it shows:

cluster services working correctly before .NET 4.8 is installed

4. Install .NET 4.8 from an offline installer. (KB4486153) and reboot.

5. After the reboot, go back to the same screen of the same Create Cluster Wizard and now it looks different:

cluster services broken afte.NET 4.8 is installed - unable to put in a 3-digit IP

Now we are unable to type in a 3 digit IP in any of the octet fields. It accepts a maximum of two characters. 

Has anyone else encountered this? It should be really easy to reproduce. 

↧

Unable to perform cluster validation -- "You do not have administrative privileges on the server ".

December 2, 2019, 8:42 am
$
0
0

I have seen this question posted on the forum many times before, but no solution worked for me.

I have three servers that I would like to configure in a failover cluster, but I cannot perform validation testing. Every time that I try to perform a validation, it gives me an error when adding a server: "You do not have administrative privileges on the server <servername>."

The account that I am using is a domain account, and it is apart of the local Admin, Remote Desktop Users, and Remote Management Users group on each server. 

To make sure that the account I was using wasn't corrupt, I deleted the user profile from the Registry and had it recreated, with no change. 

None of the servers are a DC. 

I shouldn't need to have domain permissions to create computer objects. I've done this in the past without having this ability and it worked fine. I also shouldn't need to be a domain admin. I've done this in the past without being a domain admin, and it worked fine.

I also notice that if I try to remotely manage another server through Server Manager (i.e. trying to launch "Computer Management" under a remote server), i get a popup window with the title "Event Viewer" and the message "Access Denied(5)"

It's as if the servers are not recognizing this domain account as being apart of local admins, even though it's apart of the group on each server.

↧
↧

Windows Server 2016 ESSENTIALS - Cannot create failover cluster

January 4, 2020, 9:31 am
$
0
0

Hi,

I want to make a 2 node cluster, i have already installed failover clustering feature on both the nodes through server manager.

But when i try to create a cluster, it gives me an error : " the server "node01" does not have failover feature installed. Please use server manager to install this feature"

But feature is already installed.

Can somebody help me with this error. How can i fix it.

Its urgent for me.

Regards

↧

NICs SPEED CHANGED AFTER FORMATTED

January 5, 2020, 9:57 pm
$
0
0

I have server works on win server 2012r with 4 NIC..
NIC0 = 10G speed
NIC1 = 10G speed
NIC2 = 1G speed
NIC3 = 1G speed
 when i formatted the server with HYPER-V SERVER 2019 .. All NICs are coming with same speed 1G WHY??
If that is normall with hyper-v server then what's about WINDOWS SERVER CORE 2019??

NOTE..

I DON'T TOUCH ANY UNDER INFRASTRUCTURE HARDWARE OR CABLE

↧

Will Time Difference between Cluster Nodes create issues?

November 20, 2017, 5:03 am
$
0
0

Dear All,

Just want to know if I have 2 nodes with some resources (SAN) connected and VSS enabled ,if somehow time/timezone is changed in one of the node,will it cause issues inside the cluster !!

Could you kindly enlighten me on the possibilities.

Arun

↧
Viewing all 5648 articles
Browse latest View live
Search
<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>